Image sensor, signal processing device, signal processing method, and electronic device

ABSTRACT

[Object] To execute online calibration without using a light source.[Solution] An image sensor includes: a pixel array portion in which a plurality of pixels are disposed and which generates a pixel signal; a reference signal generation unit configured to generate a reference signal for calibration; an analog digital (AD) conversion unit configured to execute AD conversion on the pixel signal and the reference signal to generate pixel data and reference data; and a correction processing unit configured to correct the pixel data on a basis of the reference data. The present technology can be applied to, for example, an image sensor performing online calibration.

CROSS REFERENCE TO RELATED APPLICATIONS

This Application claims priority under 35 U.S.C. § 119(e) to U.S.Provisional Application Ser. No. 62/696,935, entitled “IMAGING DEVICE,SIGNAL PROCESSING DEVICE, SIGNAL PROCESSING METHOD, AND ELECTRONICDEVICE,” filed on Jul. 12, 2018, which is herein incorporated byreference in its entirety.

TECHNICAL FIELD

The present technology relates to an image sensor, a signal processingdevice, a signal processing method, and an electronic device, and moreparticularly, to an image sensor, a signal processing device, a signalprocessing method, and an electronic device performing onlinecalibration.

BACKGROUND ART

In the related art, solid-state imaging devices that execute processesof calculating inflection points of all pixels using contained lightsources and execute online calibration of correcting pixel signals usingthe calculated inflection points in a case in which conditions forcausing changes in photoelectric conversion characteristics of imagesensors are satisfied have been proposed (for example, see PatentLiterature 1).

CITATION LIST Patent Literature

-   Patent Literature 1: JP 2012-244372A

DISCLOSURE OF INVENTION Technical Problem

However, when a light source is contained in a solid-state device, thesize, power consumption, and cost of the device may increase.

The present technology is devised in view of such circumstances andenables online calibration to be executed without using a light source.

Solution to Problem

An image sensor according to a first aspect of the present technologyincludes: a pixel array portion in which a plurality of pixels aredisposed and which generates a pixel signal; a reference signalgeneration unit configured to generate a reference signal forcalibration; an analog digital (AD) conversion unit configured toexecute AD conversion on the pixel signal and the reference signal togenerate pixel data and reference data; and a correction processing unitconfigured to correct the pixel data on a basis of the reference data.

A signal processing device according to a second aspect of the presenttechnology includes: a correction processing unit configured to correctpixel data obtained when an analog digital (AD) conversion unit executesAD conversion on a pixel signal generated by a pixel array portion inwhich a plurality of pixels are disposed, on a basis of reference dataobtained when the AD conversion unit executes the AD conversion on areference signal for calibration.

A signal processing method according to the second aspect of the presenttechnology includes: correcting pixel data obtained when an analogdigital (AD) conversion unit executes AD conversion on a pixel signalgenerated by a pixel array portion in which a plurality of pixels aredisposed, on a basis of reference data obtained when the AD conversionunit executes the AD conversion on a reference signal for calibration.

An electronic device according to a third aspect of the presenttechnology includes: an image sensor; and a signal processing unitconfigured to process a signal output from the image sensor. The imagesensor includes a pixel array portion in which a plurality of pixels aredisposed and which generates a pixel signal, a reference signalgeneration unit configured to generate a reference signal forcalibration, an analog digital (AD) conversion unit configured toexecute AD conversion on the pixel signal and the reference signal togenerate pixel data and reference data, and a correction processing unitconfigured to correct the pixel data on a basis of the reference data.

In the first aspect of the present technology, the pixel signal isgenerated, the reference signal for calibration is generated, the pixelsignal and the reference signal are subjected to the analog digital (AD)conversion to generate the pixel data and the reference data, and thepixel data is corrected on the basis of the reference data.

In the second aspect of the present technology, pixel data obtained whenan analog digital (AD) conversion unit executes AD conversion on a pixelsignal generated by a pixel array portion in which a plurality of pixelsare disposed is corrected, on a basis of reference data obtained whenthe AD conversion unit executes the AD conversion on a reference signalfor calibration.

In the third aspect of the present technology, the signal output fromthe image sensor is processed and the pixel signal is generated in theimage sensor, the reference signal for calibration is generated, thepixel signal and the reference signal are subjected to the analogdigital (AD) conversion to generate the pixel data and the referencedata, and the pixel data is corrected on the basis of the referencedata.

Advantageous Effects of Invention

According to the first to third aspects of the present technology, it ispossible to execute the online calibration without using a light source.

Note that the advantageous effects described in the presentspecification are merely exemplary and advantageous effects of thepresent technology are not limited to the advantageous effects describedin the present specification but additional advantageous effects may beachieved.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration example of animaging device to which the present technology is applied.

FIG. 2 is a block diagram illustrating a configuration example of animage sensor.

FIG. 3 is a diagram illustrating an example of a substrate configurationof the image sensor.

FIG. 4 is a diagram illustrating a configuration example of a pixelarray portion, an analog signal processing unit, and an AD conversionunit.

FIG. 5 is a diagram illustrating a configuration example of a pixelblock.

FIG. 6 is a diagram illustrating a configuration example of a pixel.

FIG. 7 is a diagram illustrating a configuration example of alogarithmic conversion circuit and a voltage conversion circuit.

FIG. 8 is a diagram illustrating a configuration example of a digitalsignal processing unit.

FIG. 9 is a diagram illustrating a transition method for a driving mode.

FIG. 10 is a graph illustrating examples of a light amountcharacteristic, a correction function, and a characteristic of correcteddata of a pixel block.

FIG. 11 is a graph illustrating an example of the light amountcharacteristic of each pixel block.

FIG. 12 is a diagram illustrating a temperature characteristic of thelight amount characteristic of the pixel block.

FIG. 13 is a diagram illustrating a change in corrected data during asensing mode.

FIG. 14 is a flowchart for describing an offline calibration process.

FIG. 15 is a graph illustrating a first example of the light amountcharacteristic of each pixel block.

FIG. 16 is a diagram illustrating a first example of a relation betweenthe light amount characteristic function and a reference signalcharacteristic function of the pixel block.

FIG. 17 is a diagram illustrating an example of a reference signalcharacteristic table.

FIG. 18 is a flowchart for describing an imaging process.

FIG. 19 is a flowchart for describing an online calibration process.

FIG. 20 is a diagram illustrating a first example of a relation betweenthe reference signal characteristic function and a correction functionof a pixel block.

FIG. 21 is a diagram illustrating an example of a temperaturecharacteristic of the correction function of the pixel block.

FIG. 22 is a diagram illustrating a change in corrected data during thesensing mode in a case in which the present technology is applied.

FIG. 23 is a diagram illustrating a second example of relations betweenthe light amount characteristic functions and the reference signalcharacteristic functions of the pixel blocks.

FIG. 24 is a graph illustrating a second example of the light amountcharacteristics of the pixel blocks.

FIG. 25 is a diagram illustrating a third example of relations betweenthe light amount characteristic functions and the reference signalcharacteristic functions of the pixel blocks.

FIG. 26 is a diagram illustrating a second example of relations betweenthe reference signal characteristic functions and the correctionfunctions of the pixel block.

FIG. 27 is a diagram illustrating a third example of a relation betweenthe reference signal characteristic function and the correction functionof the pixel block.

FIG. 28 is a block diagram illustrating a configuration example of atemperature detection element.

FIG. 29 is a diagram illustrating application examples of the presenttechnology.

FIG. 30 is a block diagram illustrating an example of a schematicconfiguration of a vehicle control system.

FIG. 31 is an explanatory diagram illustrating an example ofinstallation positions of a vehicle surrounding information detectionunit and an imaging unit.

MODE(S) FOR CARRYING OUT THE INVENTION

Hereinafter, a mode for carrying out the present technology (hereinafterreferred to as embodiments) will be described. Note that the descriptionthereof will be made in the following order.

1. First Embodiment

2. Second Embodiment

3. Third Embodiment

4. Fourth Embodiment

5. Modification example

6. Application example of present technology

1. First Embodiment

First, a first embodiment of the present technology will be describedwith reference to FIGS. 1 to 22.

<Configuration Example of Imaging Device 100>

FIG. 1 is a block diagram illustrating a configuration example of animaging device 100 which is an embodiment of an electronic device towhich the present technology is applied.

As the imaging device 100, for example, an information processingdevice, a smartphone, a digital camera, or the like that has an imagingfunction is assumed.

The imaging device 100 includes an optical unit 101, an image sensor102, a digital signal processor (DSP) circuit 103, a display unit 104, amanipulation unit 105, a power unit 106, a recording unit 107, a framememory 108, and a bus 109. The image sensor 102, the DSP circuit 103,the display unit 104, the manipulation unit 105, the power unit 106, therecording unit 107, and the frame memory 108 are connected to each othervia the bus 109.

The optical unit 101 condenses incident light from a subject and guidesthe incident light to the image sensor 102. The optical unit 101includes, for example, a plurality of lenses, a diaphragm, a shutter,and the like.

The image sensor 102 executes an imaging process of photoelectricallyconverting the incident light and generating image data. The imagesensor 102 supplies the generated image data (frame) to the DSP circuit103. In addition, the image sensor 102 controls the size of thediaphragm of the optical unit 101 and controls exposure of the imagingdevice 100 by supplying an exposure control signal to the optical unit101.

The DSP circuit 103 executes predetermined digital signal processing onthe image data (digital image signal) from the image sensor 102. Forexample, various kinds of digital signal processing including a demosaicprocess, a white balance process, a filtering process, and the like areexecuted. In the processes, the DSP circuit 103 causes the frame memory108 to retain the image data as necessary. Then, the DSP circuit 103supplies the processed image data to the display unit 104 or therecording unit 107.

The display unit 104 executes display or the like of an image based onthe image data.

The manipulation unit 105 generates a manipulation signal in response toa user manipulation and supplies the manipulation signal to each unit ofthe imaging device 100.

The power unit 106 supplies power to each unit of the imaging device100.

The recording unit 107 records the image data or the like.

The frame memory 108 retains the image data in units of frames.

The bus 109 is a common path used for the image sensor 102, the DSPcircuit 103, the display unit 104, the manipulation unit 105, the powerunit 106, the recording unit 107, and the frame memory 108 to exchangedata.

Note that, although the optical unit 101, the image sensor 102, the DSPcircuit 103, the display unit 104, the manipulation unit 105, the powerunit 106, the recording unit 107, and the frame memory 108 are providedin the same device in the example of FIG. 1, they may be distributed andprovided in a plurality of devices. For example, the optical unit 101 orthe image sensor 102 may be disposed in the imaging device and the DSPcircuit 103 and the like may be disposed in an information processingdevice.

<Configuration example of image sensor 102>

Next, a configuration example of the image sensor 102 of the imagingdevice 100 in FIG. 1 will be described with reference to FIGS. 2 to 8.

<Overall Configuration Example of Image Sensor 102>

FIG. 2 is a block diagram illustrating an overall configuration exampleof the image sensor 102.

The image sensor 102 includes a pixel array portion 201, a control unit202, a row scanning circuit 203, a reference signal generation unit 204,an input control unit 205, an analog signal processing unit 206, ananalog digital (AD) conversion unit 207, a column scanning circuit 208,and a digital signal processing unit 209.

In the pixel array portion 201, unit pixels including photoelectricconversion elements photoelectrically converting incident light intocharge in accordance with the amount of incident light (hereinafter alsosimply referred to as pixels) are disposed in an array form. Inaddition, in the pixel array portion 201, a pixel driving line 210 iswired for each row in the horizontal direction of the drawing (a pixelarray direction/horizontal direction of a pixel row) and a verticalsignal line 211 is wired for each column in the vertical direction ofthe drawing (a pixel array direction/vertical direction of the pixelcolumn) in the pixel arrangement of the matrix form.

One end of the pixel driving line 210 is connected to an output endcorresponding to each row of the row scanning circuit 203. Note that,although one pixel driving line 210 is illustrated for each row in FIG.2, two or more pixel driving lines 210 may be provided for each row.

The control unit 202 controls an operation of each unit of the imagingdevice 100. For example, the control unit 202 controls operation timingsof the row scanning circuit 203, the reference signal generation unit204, the input control unit 205, the analog signal processing unit 206,the AD conversion unit 207, and the column scanning circuit 208.

The row scanning circuit 203 drives each pixel of the pixel arrayportion 201 under the control of the control unit 202 so that an analogsignal with a level in accordance with an amount of incident light iscaused to be generated. In addition, the row scanning circuit 203changes a driving method for each pixel in the pixel array portion 201in accordance with four driving modes: a normal mode, a sensing mode, anoffline calibration mode, and an online calibration mode, under thecontrol of the control unit 202.

The normal mode is a mode in which image data with a high resolution(hereinafter referred to as high-resolution image data) is generated onthe basis of a pixel signal of each pixel of the pixel array portion201. In the normal mode, the analog pixel signal output from each pixelof the pixel array portion 201 is supplied from the pixel array portion201 to the input control unit 205.

The sensing mode is a mode in which image data with a low resolution(hereinafter referred to as low-resolution image data) is generated onthe basis of an added signal which is a pixel signal obtained by addingsignals of a plurality of pixels for each pixel block of a predeterminedunit of the pixel array portion 201 and an event is detected on thebasis of the low-resolution image data. In the sensing mode, an analogadditional pixel output from each pixel block of the pixel array portion201 is supplied from the pixel array portion 201 to the input controlunit 205.

The offline calibration mode is a mode in which offline calibration ofthe image sensor 102 is executed, as will be described below. In theoffline calibration mode, each pixel in the pixel array portion 201 isdriven in accordance with a similar method to the sensing mode and theanalog added signal is supplied from the pixel array portion 201 to theinput control unit 205.

The online calibration mode is a mode in which online calibration of theimage sensor 102 is executed, as will be described below. In the onlinecalibration mode, each pixel in the pixel array portion 201 is driven inaccordance with a similar method to the sensing mode and the analogadded signal is supplied from the pixel array portion 201 to the inputcontrol unit 205.

The reference signal generation unit 204 generates a reference signal tobe used for calibration instead of the added signal and supplies thereference signal to the input control unit 205 under the control of thecontrol unit 202.

The input control unit 205 includes, for example, a switch or the likeand selects a signal to be input to the analog signal processing unit206 under the control of the control unit 202. Specifically, the inputcontrol unit 205 selects one of the analog signal (the pixel signal orthe added signal) from the pixel array portion 201 and the referencesignal from the reference signal generation unit 204 and supplies theselected signal to the analog signal processing unit 206.

The analog signal processing unit 206 executes predetermined analogsignal processing on the added signal and the reference signal andsupplies the added signal and the reference signal to the AD conversionunit 207. For example, the analog signal processing unit 206 executes alogarithmic conversion process and a voltage conversion process on theadded signal and the reference signal.

Note that the analog signal processing unit 206 supplies the pixelsignal to the AD conversion unit 207 without particularly processing thepixel signal.

The AD conversion unit 207 executes AD conversion on the analog signalfrom the analog signal processing unit 206. Specifically, the ADconversion unit 207 executes the AD conversion on the analog pixelsignal, added signal, and reference signal to generate digital pixeldata, added data, and reference data. The AD conversion unit 207supplies the generated pixel data, added data, and reference data to thedigital signal processing unit 209.

Note that since the added signal is a signal obtained by adding thepixel signals of the plurality of pixels, the added data is pixel dataobtained by adding the pixel data of the plurality of pixels.

The column scanning circuit 208 includes a shift register, an addressdecoder, or the like and selectively scans a circuit portioncorresponding to a pixel column of the pixel array portion 201. Thus,signal processing is executed on the analog signals (the pixel signal,the added signal, and the reference signal) for each selected column andthe processed signals are supplied to the digital signal processing unit209.

The digital signal processing unit 209 executes predetermined digitalsignal processing on the pixel data, the added data, and the referencedata.

For example, the digital signal processing unit 209 generateshigh-resolution image data on the basis of the pixel data of each pixelof the pixel array portion 201 and supplies the generatedhigh-resolution image data to the DSP circuit 103.

For example, the digital signal processing unit 209 generateslow-resolution image data on the basis of the added data of each pixelblock of the pixel array portion 201 and executes an event detectionprocess on the basis of the low-resolution image data. Note that anyevent which is a detection target is set. For example, motion detection,face detection, human form detection, or the like within an imagingrange of the imaging device 100 is executed as event detection. Thedigital signal processing unit 209 sets a driving mode on the basis of adetection result or the like of the event and supplies a driving modesignal indicating the set driving mode to the control unit 202.

For example, the digital signal processing unit 209 generates anexposure control signal for controlling exposure of the imaging device100 on the basis of the added data and supplies the exposure controlsignal to the optical unit 101 and the control unit 202.

For example, the digital signal processing unit 209 executes calibrationof a correction function for correction of the added data on the basisof the added data and the reference data.

<Substrate Configuration Example of Image Sensor 102>

FIG. 3 is a diagram illustrating an example of a substrate configurationof the image sensor 102.

FIG. 3A illustrates a first substrate configuration of the image sensor102. In an image sensor 102 a in FIG. 3A, a pixel region 261, a controlregion 262, and a logic circuit 263 that includes a signal processingcircuit executing signal processing are provided in one semiconductorsubstrate 251. In the pixel region 261, for example, the pixel arrayportion 201 in FIG. 2 is provided. In the control region 262, forexample, the control unit 202, the row scanning circuit 203, thereference signal generation unit 204, the input control unit 205, theanalog signal processing unit 206, the AD conversion unit 207, and thecolumn scanning circuit 208 in FIG. 2 are provided. In the logic circuit263, for example, the digital signal processing unit 209 in FIG. 2 isprovided.

FIGS. 3B and 3C illustrate second and third substrate configurations ofthe image sensor 102. In an image sensor 102 b in FIG. 3B and an imagesensor 102 c in FIG. 3C, the pixel region 261 and the logic circuit 263are formed on different semiconductor substrates to realize a stackedstructure.

In the image sensor 102 b, the pixel region 261 and the control region262 are provided in a first semiconductor substrate 252, and the logiccircuit 263 is provided in a second semiconductor substrate 253. Thefirst semiconductor substrate 252 and the second semiconductor substrate253 are electrically connected to each other.

In the image sensor 102 c, the pixel region 261 is provided in the firstsemiconductor substrate 252, and the control region 262 and the logiccircuit 263 are provided in the second semiconductor substrate 253. Thefirst semiconductor substrate 252 and the second semiconductor substrate253 are electrically connected to each other.

JP 2010-245506A, JP 2011-96851A, and the like by the present applicantsdisclose methods of manufacturing a solid-state imaging device in whichthe first semiconductor substrate 252 in which the pixel region 261 isformed and the second semiconductor substrate 253 in which the logiccircuit 263 is formed are separately formed using a semiconductorprocess technology and are pasted to be electrically connected to eachother, as in the image sensor 102 b and the image sensor 102 c. Byforming the semiconductor substrates separately and pasting themtogether, contribution to high image quality, mass productivity, and lowcost can be achieved. In addition, for example, the first semiconductorsubstrate 252 and the second semiconductor substrate 253 can bemanufactured at different locations.

<Detailed Configuration Example of Each Unit of Image Sensor 102>

Next, detailed configuration examples of the pixel array portion 201,the analog signal processing unit 206, the AD conversion unit 207, andthe digital signal processing unit 209 of the image sensor 102 will bedescribed with reference to FIGS. 4 to 8.

FIG. 4 is a diagram illustrating a detailed configuration example of thepixel array portion 201, the analog signal processing unit 206, and theAD conversion unit 207. Note that in FIG. 4, the control unit 202, therow scanning circuit 203, the reference signal generation unit 204, theinput control unit 205, the column scanning circuit 208, and the digitalsignal processing unit 209 are not illustrated.

In the pixel array portion 201, pixels 301 are disposed in an arrayform. In the array of the pixels 301 with the matrix form, the verticalsignal line 211 is formed in the vertical direction for each column.

In addition, in the pixel array portion 201, as illustrated in FIG. 5,the pixels 301 are divided into a plurality of pixel blocks PB. Then, aswill be described below, an added signal is generated by adding thepixel signals of the pixels 301 in the pixel blocks PB.

Note that, although one pixel block PB is formed by the pixels 301 in 8vertical rows×8 horizontal columns in this example, the pixel block PBcan be set to have any size.

In addition, hereinafter, in a case in which it is necessary toindividually distinguish the pixel blocks PB, numbers are suffixed toreference signs PB as in a pixel block PB1 and a pixel block PB2.

FIG. 6 is a diagram illustrating an expanded configuration example ofthe pixel 301.

The pixel 301 has a coexistent pixel structure of 2 horizontal rows×2vertical columns. Specifically, the pixel 301 includes photoelectricconversion elements 341-1 to 341-4 and transfer gate portions 342-1 to342-4.

Note that, hereinafter, in a case in which it is not necessary toindividually distinguish the photoelectric conversion elements 341-1 to341-4, the photoelectric conversion elements 341-1 to 341-4 are simplyreferred to as the photoelectric conversion elements 341. Hereinafter,in a case in which it is not necessary to individually distinguish thetransfer gate portions 342-1 to 342-4, the transfer gate portions 342-1to 342-4 are simply referred to as the transfer gate portions 342.

Then, the four photoelectric conversion elements 341 and the fourtransfer gate portions 342 share one charge voltage conversion portion343, a reset transistor 344, an amplification transistor 345, a selecttransistor 346, and a coupling transistor 347.

Note that, hereinafter, the reset transistor 344, the amplificationtransistor 345, the select transistor 346, and the coupling transistor347 are collectively referred to as pixel transistors.

Each photoelectric conversion element 341 includes, for example, a PNjunction photodiode, receives light from a subject, and generates andstores charge in accordance with the amount of received light (amount ofincident light) by photoelectric conversion.

The transfer gate unit 342-1 includes, for example, an N-channel MOStransistor and is provided between the photoelectric conversion element341-1 and the charge voltage conversion portion 343. A driving signalTRG1 is supplied to a gate of the transfer gate portion 342-1. Thedriving signal TRG1 is a pulse signal in which a high level state is anactive state (ON state) and a low level state is an inactive state (OFFstate). Then, when the driving signal TRG1 enters the active state andthe transfer gate portion 342-1 is turned on (enters a conductivestate), the charge stored in the photoelectric conversion element 341-1is transferred to the charge voltage conversion portion 343 via thetransfer gate portion 342-1.

Similarly, the transfer gate units 342-2 to 342-4 include N-channel MOStransistors and are provided between the photoelectric conversionelements 341-2 to 341-4 and the charge voltage conversion portion 343,respectively. Similarly to the transfer gate portion 342-1, the transfergate portions 342-2 to 342-4 transfer charge stored in the photoelectricconversion elements 341-2 to 341-4 to the charge voltage conversionportion 343 in accordance with driving signals TRG2 to TRG4 supplied tothe respective gates.

The charge voltage conversion portion 343 is a floating diffusion region(FD) that converts the charge transferred from each photoelectricconversion element 341 via each transfer gate portion 342 into anelectric signal (for example, a voltage signal) and outputs the electricsignal. The charge voltage conversion portion 343 is connected to thereset transistor 344 and the coupling transistor 347 and is connected tothe vertical signal line 211 via the amplification transistor 345 andthe select transistor 346.

The reset transistor 344 is an element that appropriately initializes(resets) the charge voltage conversion portion 343 or the like andincludes, for example, an N-channel MOS transistor. A drain of the resettransistor 344 is connected to a power supply VDD of a voltage VDD via apower line and a source of the reset transistor 344 is connected to thecharge voltage conversion portion 343. A driving signal RST is appliedas a reset signal to a gate of the reset transistor 344. The drivingsignal RST is a pulse signal in which a high level state is an activestate (ON state) and a low level state is an inactive state (OFF state).Then, when the driving signal RST enters the active state, the resettransistor 344 is turned on and a potential of the charge voltageconversion portion 343 or the like is reset to the voltage VDD. That is,the charge voltage conversion portion 343 or the like is initialized.

The amplification transistor 345 includes, for example, an N-channel MOStransistor. A gate of the amplification transistor 345 is connected tothe charge voltage conversion portion 343 and a drain of theamplification transistor 345 is connected to the power supply VDD andserves as an input unit of a source follower circuit that reads chargeobtained through the photoelectric conversion in the photoelectricconversion element 341. That is, a source of the amplificationtransistor 345 is connected to the vertical signal line 211 via theselect transistor 346, and thus the amplification transistor 345 formsthe source follower circuit along with a constant current source 302(see FIG. 4) connected to one end of the vertical signal line 211.

The select transistor 346 includes, for example, an N-channel MOStransistor and is connected between the source of the amplificationtransistor 345 and the vertical signal line 211. A driving signal SEL issupplied as a select signal to a gate of the select transistor 346. Thedriving signal SEL is a pulse signal in which a high level state is anactive state (ON state) and a low level state is an inactive state (OFFstate). Then, when the driving signal SEL enters the active state, theselect transistor 346 is turned on and the pixel 301 in which the selecttransistor 346 is provided enters a select state. When the pixel 301enters the select state, a signal (pixel signal) output from theamplification transistor 345 is supplied to the input control unit 205via the vertical signal line 211.

The coupling transistor 347 is a switch for connecting the pixel 301 inthe same pixel block PB and includes, for example, an N-channel MOStransistor. A drain of the coupling transistor 347 is connected to thecharge voltage conversion portion 343 and a source of the couplingtransistor 347 is connected to a source (see FIGS. 4 and 7) of thelogarithmic conversion transistor 361 of the logarithmic conversioncircuit 311 via the input control unit 205. A driving signal LOGEN isapplied to a gate of the coupling transistor 347. The driving signalLOGEN is a pulse signal in which a high level state is an active state(ON state) and a low level state is an inactive state (OFF state).

Then, when the driving signal LOGEN of each pixel 301 in the pixel blockPB enters the active state, the coupling transistor 347 is turned on,and thus the pixels 301 in the pixel block PB are connected. At thistime, when the driving signals TRG1 to TRG4 of each pixel 301 enter theactive state, the charge stored in each photoelectric conversion element341 of each pixel 301 in the pixel block PB is added, and thus an addedsignal obtained by adding the pixel signals of the pixels 301 in thepixel block PB is generated. The generated added signal is supplied tothe input control unit 205.

For example, in the sensing mode, the transfer gate portion 342 and thecoupling transistor 347 of each pixel 301 enter the normally ON stateand an added signal is normally output from each pixel block PB.

In each pixel 301, as the pixel driving lines 210 in FIG. 1, a pluralityof driving lines are wired, for example, for each row. Then, the drivingsignals TRG1 to TRG4, the driving signal RST, the driving signal SEL,and the driving signal LOGEN are supplied from the row scanning circuit203 to each pixel 301 via the plurality of driving signals serving asthe pixel driving lines 210.

Referring back to FIG. 4, in the analog signal processing unit 206, forexample, the logarithmic conversion circuit 311 and the voltageconversion circuit 312 are individually provided for each pixel blockPB.

FIG. 7 is a diagram illustrating a configuration example of thelogarithmic conversion circuit 311 and the voltage conversion circuit312.

The logarithmic conversion circuit 311 includes the logarithmicconversion transistor 361. The logarithmic conversion transistor 361includes, for example, an N-channel MOS transistor. A drain and a gateof the logarithmic conversion transistor 361 are connected to the powersupply VDD, and a source of the logarithmic conversion transistor 361 isconnected to the reference signal generation unit 204 and the source ofthe coupling transistor 347 of each pixel 301 in the corresponding pixelblock PB via the input control unit 205.

The logarithmic conversion circuit 311 executes a logarithmic conversionprocess on the added signal supplied from the input control unit 205 andsupplies the processed added signal to the voltage conversion circuit312. Thus, the added signal of each pixel block PB is a signal that islogarithmically changed in accordance with an amount of light incidenton the pixel block PB. In addition, the logarithmic conversion circuit311 executes the logarithmic conversion process on a reference signalsupplied from the input control unit 205 and supplies the processedreference signal to the voltage conversion circuit 312.

The voltage conversion circuit 312 includes a transfer gate portion 381,an amplification transistor 382, and a select transistor 383.

The transfer gate portion 381 includes, for example, an N-channel MOStransistor. A drain of the transfer gate portion 381 is connected to thesource of the logarithmic conversion transistor 361 and a drain of thetransfer gate portion 381 is connected to the gate of the amplificationtransistor 382. A driving signal LOGTRG is supplied to a gate of thetransfer gate portion 381. The driving signal LOGTRG is a pulse signalin which a high level state is an active state (ON state) and a lowlevel state is an inactive state (OFF state). Then, when the drivingsignal LOGTRG enters the active state and the transfer gate portion 381is turned on (enters a conductive state), the reference signal or theadded signal supplied from the logarithmic conversion circuit 311 isinput to a gate of the amplification transistor 382.

The amplification transistor 382 includes, for example, an N-channel MOStransistor. A drain of the amplification transistor 345 is connected tothe power supply VDD and serves as an input unit of a source followercircuit that reads the added signal or the reference signal (charge ofthe added signal or the reference signal). That is, a source of theamplification transistor 382 is connected to any one of the verticalsignal lines 211 in the pixel block PB via the select transistor 383,and thus the amplification transistor 382 forms the source followercircuit along with a constant current source 302 (see FIG. 4) connectedto one end of the vertical signal line 211.

The select transistor 383 includes, for example, an N-channel MOStransistor and is connected between the source of the amplificationtransistor 382 and the vertical signal line 211. A driving signal LOGSELis supplied as a select signal to a gate of the select transistor 383.The driving signal LOGSEL is a pulse signal in which a high level stateis an active state (ON state) and a low level state is an inactive state(OFF state). Then, when the driving signal LOGSEL enters the activestate, the select transistor 383 is turned on and the pixel block PB inwhich the select transistor 383 is provided enters a select state. Whenthe pixel block PB enters the select state, a signal (added signal orreference signal) output from the amplification transistor 382 issupplied to the input control unit 205 via the vertical signal line 211.

Note that, for example, the driving signals LOGTRG and LOGSEL aresupplied from the control unit 202 to each voltage conversion circuit312.

Referring back to FIG. 4, in the AD conversion unit 207, an analogdigital converter (ADC) 321 is provided in each vertical signal line211. Each ADC 321 executes AD conversion on the pixel signal, the addedsignal, and the reference signal supplied via the vertical signal line211, generates pixel data, added data, and reference data, and suppliesthe pixel data, the added data, and the reference data to the digitalsignal processing unit 209.

<Configuration Example of Digital Signal Processing Unit 209>

FIG. 8 is a block diagram illustrating a configuration example of thedigital signal processing unit 209.

The digital signal processing unit 209 includes an input control unit401, a correction processing unit 402, an event detection unit 403, amode control unit 404, an exposure control unit 405, an image processingunit 406, and a storage unit 407.

The input control unit 401 includes, for example, a switch or the likeand controls input destinations of the pixel data, the added data, andthe reference data supplied from the AD conversion unit 207. Forexample, the input control unit 401 supplies the pixel data to the imageprocessing unit 406 and supplies the added data and the reference datato the correction processing unit 402.

The correction processing unit 402 executes a process of correcting theadded data on the basis of the reference data. The correction processingunit 402 includes a calibration unit 411 and a correction unit 412.

The calibration unit 411 calibrates a correction function for correctionof the added data. The calibration unit 411 includes a light amountcharacteristic estimation unit 421, a reference signal characteristicestimation unit 422, and a correction function generation unit 423.

On the basis of a measurement result of the added data with regard to apredetermined amount of incident light, the light amount characteristicestimation unit 421 estimates a light amount characteristic indicating arelation between the added data and the amount of incident light of eachpixel block PB.

The reference signal characteristic estimation unit 422 estimates areference signal characteristic indicating a correspondent relationbetween the reference signal and the amount of incident light of eachpixel block PB on the basis of the measurement result of the referencedata and the light amount characteristic of each pixel block PB. Thereference signal characteristic estimation unit 422 generates areference signal characteristic table indicating a reference signalcharacteristic of each pixel block PB and causes a nonvolatile memory431 of the storage unit 407 to store the reference signal characteristictable.

The correction function generation unit 423 generates a correctionfunction with regard to each pixel block PB on the basis of themeasurement result of the reference data and the reference signalcharacteristic of each pixel block PB. The correction function correctsthe added data logarithmically changed with respect to the amount ofincident light so that the added data is linearly changed with respectto the amount of incident light. That is, the correction functioncorrects a relation between the added data and the amount of incidentlight to a linear relation. In addition, the correction functioncorrects a variation in the light amount characteristic between thepixel blocks PB.

The correction unit 412 corrects the added data on the basis of thecorrection function. The correction unit 412 supplies the correctedadded data (hereinafter referred to as corrected data) to the eventdetection unit 403 and the exposure control unit 405.

The event detection unit 403 detects a predetermined event on the basisof low-resolution image data generated from the corrected data. Theevent detection unit 403 supplies data indicating a detection result ofthe event to the mode control unit 404 and the exposure control unit405.

The mode control unit 404 controls (switches) a driving mode on thebasis of the detection result or the like of the event. The mode controlunit 404 supplies a mode signal indicating the set driving mode to theinput control unit 401, the correction processing unit 402, and thecontrol unit 202 (see FIG. 2).

The exposure control unit 405 executes exposure control of the imagingdevice 100 on the basis of the corrected data. The exposure control unit405 generates an exposure control signal for executing exposure controlof the imaging device 100 and supplies the exposure control signal tothe optical unit 101 (see FIG. 1) and the control unit 202 (see FIG. 2).

The image processing unit 406 generates high-resolution image data onthe basis of the pixel data and supplies the generated high-resolutionimage data to the DSP circuit 103.

The storage unit 407 includes a nonvolatile memory 431 and a volatilememory 432. The nonvolatile memory 431 includes, for example, a one-timeprogrammable (OTP) memory or a rewritable nonvolatile memory such as aflash memory and stores data necessary for preservation, such as areference signal characteristic table. The volatile memory 432 includesa volatile memory such as an SRAM and stores data to be temporarily usedin a process of the digital signal processing unit 209.

<Necessity of Calibration>

Next, necessity of calibration of the imaging device 100 will bedescribed with reference to FIGS. 9 to 12.

As illustrated in FIG. 9, the imaging device 100 transitions to thenormal mode when an event is detected on the basis of the low-resolutionimage data in the sensing mode. Then, in the normal mode, the normalmode transitions to the sensing mode after image data (high-resolutionimage data) necessary for analysis or the like of a generated event isacquired.

Here, before the sensing mode transitions to the normal mode, automaticexposure control (AE) is executed such that the high-resolution imagedata can be acquired with appropriate exposure from the first frame. Theautomatic exposure control is executed on the basis of the correcteddata.

FIG. 10 illustrates a relation among illumination X, the added data x,and corrected data y in the pixel block PB.

FIG. 10A illustrates a graph of a light amount characteristic indicatinga relation between the illumination Y in the pixel block PB and theadded data x based on the added signal output from the pixel block PB.The horizontal axis represents the illumination y (of which a unit is alux) and the vertical axis represents the added data x.

Note that an amount of light incident on the pixel block PB iscalculated by multiplying the illumination Y by the area of a lightreception surface of the pixel block PB. Accordingly, the relationbetween the illumination Y of the pixel block PB and the added data xindicates a relation between the amount of incident amount of the pixelblock PB and the added data x.

As described above, since the added signal is a signal logarithmicallyconverted by the logarithmic conversion circuit 311, the added data x islogarithmically converted without being linearly changed with respect tothe illumination Y. Accordingly, it is necessary to execute correctionusing the correction function so that the added data x is linearlychanged with respect to the illumination Y.

FIG. 10B is a graph illustrating an example of a correction function.The horizontal axis represents the added data x and the vertical axisrepresents the corrected data y.

FIG. 10C is a graph illustrating a relation between the illumination Yand the corrected data y in a case in which the correction function ofFIG. 10B is used. The horizontal axis represents the illumination y (ofwhich a unit is a lux) and the vertical axis represents the correcteddata y. As illustrated in the graph, the corrected data y linearlychanged with respect to the illumination Y can be obtained by correctingthe added data x using the correction function.

Here, the light amount characteristic of each pixel block PB has avariation between the pixel blocks PB.

For example, FIG. 11 illustrates a graph of a light amountcharacteristic as in FIG. 10A. Specifically, FIG. 11 illustrates a graphof a light amount characteristic of each pixel block PB. The lightamount characteristics of the pixel blocks PB are indicated by differentkinds of curve lines.

The variation in the light amount characteristic between the pixelblocks PB occurs due to a variation in the characteristic of each pixel301, a variation in the characteristics of the logarithmic conversioncircuit 311, the voltage conversion circuit 312, and the ADC 321corresponding to each pixel block PB, or the like. A variation in acharacteristic of each pixel 301 occurs due to, for example, a variationin sensitivity of the photoelectric conversion element 341.

Accordingly, it is preferable to correct the variation in the lightamount characteristic between the pixel blocks PB by correcting theadded data to corrected data using a different correction function foreach pixel block PB (for each added data).

In addition, the light amount characteristic of each pixel block PBchanges in accordance with ambient temperature of the imaging device100.

FIG. 12 is a diagram illustrating an example of a temperaturecharacteristic of the light amount characteristic of the pixel block PB.FIG. 12A illustrates an example of a change of the graph of the lightamount characteristic in accordance with the temperature. The horizontalaxis represents the illumination Y (of which a unit is a lux) and thevertical axis represents the added data x. FIG. 12B illustrates anexample of a change of the added data in accordance with the temperaturein a case in which the illumination is constant in the pixel block PB.The horizontal axis represents a temperature (of which a unit is ° C.)and the vertical axis represents the added data x.

As illustrated in FIG. 12A, the graph of the light amount characteristicis moved in an arrow direction as the ambient temperature increases. Inaddition, as illustrated in FIG. 12B, even when the illumination (theamount of incident light) is constant in the pixel block PB, the addeddata x decreases with an increase in the ambient temperature.

Note that the characteristic of each pixel 301 is rarely affected by thetemperature and the change in the characteristic in accordance with thetemperature is small. On the other hand, the characteristics of thelogarithmic conversion circuit 311 and the voltage conversion circuit312 of the analog signal processing unit 206 and the ADC 321 of the ADconversion unit 207 are easily affected by the temperature and thechange in the characteristics in accordance with the temperature islarge. In particular, the characteristic of the logarithmic conversioncircuit 311 considerably changes in accordance with the temperature.

Accordingly, the temperature change of the light amount characteristicof each pixel block PB considerably results from the circuit after theadded signal is output from each pixel block PB.

The lower drawing of FIG. 13 illustrates a graph indicating an exampleof a change in the corrected data y with respect to the same amount ofincident light of that during the sensing mode. The horizontal axisrepresents a time and the vertical axis represents the corrected data y.

For example, although the amount of light incident on the pixel block PBis constant, the light amount characteristic is changed when the ambienttemperature changes. Therefore, a value of the added data x changes.Accordingly, a value of the corrected data y obtained by correcting theadded data x also changes.

For example, a difference Δy occurs between the value of the correcteddata y at time t0 at the time of starting of the sensing mode and thecorrected data y at time t1 at the time of ending of the sensing mode.

Accordingly, in a case in which the automatic exposure control isexecuted on the basis of the corrected data y, a difference occurs in acontrol amount of exposure between time t0 and time t1 even when theamount of incident light (brightness of a subject or a background) isthe same. As a result, for example, even when the exposure isappropriately set at time t0, the exposure is not appropriately set attime t1 in some cases. Thus, the exposure of the imaging device 100 isnot appropriately set irrespective of the automatic exposure controlexecuted before transition to the normal mode by detecting an event, andthus there is concern of quality of the high-resolution image data atthe time of starting of the normal mode deteriorating.

In addition, when a characteristic of the corrected data y changes inaccordance with the ambient temperature during the sensing mode, qualityof the low-resolution image data deteriorates. As a result, there isconcern of detection precision of an event deteriorating.

In contrast, the imaging device 100 appropriately calibrates thecorrection function (online calibration) during imaging. Thus, the lightamount characteristic of each pixel block PB is maintained substantiallyconstantly irrespective of a change in temperature and a variationbetween the pixel blocks PB is suppressed.

<Process of Imaging Device 100>

Next, a process of the imaging device 100 will be described withreference to FIGS. 14 to 22.

<Offline Calibration Process>

First, an offline calibration process executed by the imaging device 100will be described with reference to the flowchart of FIG. 14.

This process is executed only once in a test step after manufacturing,for example, in a case in which the nonvolatile memory 431 of the imagesensor 102 is of an OTP type. This process is executed at apredetermined timing (for example, periodically once per year) in a casein which the nonvolatile memory 431 is of a rewritable type.

In step S1, the mode control unit 404 is set to the offline calibrationmode. The mode control unit 404 supplies a mode signal indicating theset offline calibration mode to the input control unit 401, thecorrection processing unit 402, and the control unit 202.

Thus, a process of outputting the added data starts. Specifically, thefollowing process starts.

The pixel array portion 201 generates the added signal of each pixelblock PB and supplies the added signal to the input control unit 205.

The input control unit 205 supplies the added signal of each pixel blockPB to the analog signal processing unit 206.

The analog signal processing unit 206 executes the logarithmicconversion and the voltage conversion on the added signal of each pixelblock PB and supplies the converted added signal to the AD conversionunit 207.

The AD conversion unit 207 converts the analog added signal of eachpixel block PB into the digital added data and supplies the added dataof each pixel block PB to the input control unit 401 of the digitalsignal processing unit 209.

The input control unit 401 supplies the added data of each pixel blockPB to the correction processing unit 402.

In step S2, the imaging device 100 measures the added data with regardto the amount of incident light with a low level.

For example, the light reception surface of the pixel array portion 201of the image sensor 102 is set substantially uniformly with illuminationYL of a predetermined low level using an external light source. In thisstate, the imaging device 100 executes an imaging process, for example,N times. Then, the light amount characteristic estimation unit 421acquires the added data of each block PB equivalent to N frames andretains the added data in the volatile memory 432.

In step S3, the light amount characteristic estimation unit 421calculates an average value of the added data with regard to the amountof incident light with the low level. Specifically, the light amountcharacteristic estimation unit 421 calculates an average value of theadded data of each pixel block PB equivalent to N frames obtained in theprocess of step S2. The light amount characteristic estimation unit 421causes the volatile memory 432 to retain the average value of the addeddata of each pixel block PB.

In step S4, the imaging device 100 measures the added data with regardto the amount of incident light with a middle level. That is, in a statein which the light reception surface of the pixel array portion 201 ofthe image sensor 102 is set substantially uniformly with illumination YM(>illumination YL) of a predetermined middle level using an externallight source, a process similar to step S2 is executed.

In step S5, through a process similar to step S3, an average value ofthe added data of each pixel block PB with regard to the amount ofincident light with the middle level is calculated and retained in thevolatile memory 432.

In step S6, the imaging device 100 measures the added data with regardto the amount of incident light with a high level. That is, in a statein which the light reception surface of the pixel array portion 201 ofthe image sensor 102 is set substantially uniformly with illumination YH(>illumination YM) of a predetermined high level using an external lightsource, a process similar to step S2 is executed.

In step S7, through a process similar to step S3, an average value ofthe added data of each pixel block PB with regard to the amount ofincident light with the high level is calculated and retained in thevolatile memory 432.

In step S8, the light amount characteristic estimation unit 421estimates the light amount characteristic of the image sensor 102 on thebasis of a measurement result of the added data.

For example, the light amount characteristic estimation unit 421generates a quadratic interpolation function on the basis of the averagevalue of the added data with regard to the illumination at three stagesof the illumination YL, the illumination YM, and the illumination YH foreach pixel block PB. The interpolation function is a function thatexpresses the light amount characteristic indicating the relationbetween the added data and the illumination (that is, the amount ofincident light) of each pixel block PB and is referred to as a lightamount characteristic function.

FIG. 15 is a graph illustrating an example of the light amountcharacteristic function. The horizontal axis represents added data andthe vertical axis represents illumination (of which a unit is a lux).

Points P1L, P1M, and P1H are points corresponding to the pieces of addeddata (an average value of the added data) x1L, x1M, and x1H of the pixelblock PB1 in the illumination YL, the illumination YM, and theillumination YH. Then, a light amount characteristic function fa1 a ofthe pixel block PB1 is generated through quadratic interpolation basedon the points P1L, P1M, and P1H.

Points P2L, P2M, and P2H are points corresponding to the pieces of addeddata (an average value of the added data) x2L, x2M, and x2H of the pixelblock PB2 in the illumination YL, the illumination YM, and theillumination YH. Then, a light amount characteristic function fa2 a ofthe pixel block PB2 is generated through quadratic interpolation basedon the points P2L, P2M, and P2H.

In this way, the light amount characteristic function of each pixelblock PB is generated. Then, the light amount characteristic estimationunit 421 causes the volatile memory 432 to retain data (for example, acoefficient of the light amount characteristic function) indicating thequadratic light amount characteristic function of each pixel block PB.

At this time, a measurement result of the added data used to generatethe light amount characteristic function of each pixel block PB iserased from the volatile memory 432.

In step S9, the input control unit 205 switches the signal to thereference signal. Specifically, the row scanning circuit 203 turns thedriving signals TRG1 to TRG4 of each pixel 301 and the driving signalLOGEN off under the control of the control unit 202. Thus, the output ofthe added signal from each pixel block PB is stopped.

The input control unit 205 switches the signal supplied to the analogsignal processing unit 206 from the added signal of each pixel block PBsupplied from the pixel array portion 201 to the reference signalsupplied from the reference signal generation unit 204 under the controlof the control unit 202.

In step S10, the imaging device 100 measures the reference data withregard to the reference signal with the low level.

Specifically, the reference signal generation unit 204 generates areference signal with a current value iL of a predetermined low leveland supplies the reference signal to the input control unit 205 underthe control of the control unit 202.

The input control unit 205 supplies the reference signal to thelogarithmic conversion circuit 311 corresponding to each pixel block PBin the analog signal processing unit 206 under the control of thecontrol unit 202.

The logarithmic conversion circuit 311 and the voltage conversioncircuit 312 corresponding to each pixel block PB execute the logarithmicconversion and the voltage conversion and supply the processed referencesignal to the AD conversion unit 207.

Note that, hereinafter, the reference signal processed through thelogarithmic conversion and the voltage conversion by the logarithmicconversion circuit 311 and the voltage conversion circuit 312corresponding to each pixel block PB is referred to as a referencesignal corresponding to each pixel block PB or a reference signal ofeach pixel block PB.

The ADC 321 corresponding to each pixel block PB executes the ADconversion on the analog reference signal, generates digital referencedata, and supplies the converted reference data to the correctionprocessing unit 402 via the input control unit 205 of the digital signalprocessing unit.

Accordingly, the reference data of each pixel block PB becomes dataobtained by converting the reference signal into the reference datausing the logarithmic conversion circuit 311, the voltage conversioncircuit 312, and the ADC 321 corresponding to each pixel block PB.

In this state, for example, the reference signal characteristicestimation unit 422 executes measurement of the reference data of eachpixel block PB N times and causes the volatile memory 432 to retain ameasurement result.

In step S11, the reference signal characteristic estimation unit 422calculates the average value of the reference data with regard to thereference signal with the low level. Specifically, the reference signalcharacteristic estimation unit 422 calculates the average value of thereference data of each pixel block PB measured in the process of stepS10. The reference signal characteristic estimation unit 422 causes thevolatile memory 432 to retain the average value of the reference data ofeach pixel block PB.

In step S12, the imaging device 100 measures each piece of referencedata with regard to the reference signal with the middle level. That is,in a state in which the current value of the reference signal is set toa current value iM (>the current value iL) of the predetermined middlelevel, a process similar to step S10 is executed.

In step S13, through a process similar to step S11, the average value ofthe reference data of each pixel block PB with regard to the referencesignal with the middle level is calculated and retained in the volatilememory 432.

In step S14, the imaging device 100 measures each piece of referencedata with regard to the reference signal with the high level. That is,in a state in which the current value of the reference signal is set toa current value iH (>the current value iM) with the predetermined highlevel, a process similar to step S10 is executed.

In step S15, through a process similar to step S11, the average value ofthe reference data of each pixel block PB with regard to the referencesignal with the high level is calculated and retained in the volatilememory 432.

In step S16, the reference signal characteristic estimation unit 422estimates the reference signal characteristic of the image sensor 102 onthe basis of the light amount characteristic and the measurement resultof the reference data.

Specifically, the reference signal characteristic estimation unit 422estimates illumination of each pixel block PB equivalent to thereference signal with each level by substituting the reference data (theaverage value of the reference data) with regard to the reference signalwith each level of each pixel block PB into the light amountcharacteristic function of each pixel block PB.

Here, a specific example of the process of step S16 will be describedwith reference to FIG. 16.

FIG. 16B illustrates graphs of the same light amount characteristicfunction fa1 a and light amount characteristic function fa2 a as thoseof FIG. 15. FIG. 16A illustrates a graph of a reference signalcharacteristic indicating a correspondent relation between the referencesignal and the illumination with regard to the pixel block PB. Thehorizontal axis represents a current value (of which a unit is A) of thereference signal and the vertical axis represents illumination (of whicha unit is a lux) of the pixel block PB.

For example, in a case in which the reference data (the average value ofthe reference data) of the pixel block PB1 with regard to the referencesignal with the low level is x11L, as illustrated in FIG. 16B,illumination YL1 is calculated by substituting reference data x11L intothe light amount characteristic function fa1 a. Then, as illustrated inFIGS. 16A and 16B, the current value iL and the illumination YL1 of thereference signal with the low level are associated. Thus, theillumination of the pixel block PB1 equivalent to the reference signalwith the low level is estimated to be the illumination YL1.

For example, in a case in which the reference data (the average value ofthe reference data) of the pixel block PB1 with regard to the referencesignal with the middle level is x11M, as illustrated in FIG. 16B,illumination YM1 is calculated by substituting reference data x11M intothe light amount characteristic function fa1 a. Then, as illustrated inFIGS. 16A and 16B, the current value iM and the illumination YM1 of thereference signal with the middle level are associated. Thus, theillumination of the pixel block PB1 equivalent to the reference signalwith the middle level is estimated to be the illumination YM1.

For example, in a case in which the reference data (the average value ofthe reference data) of the pixel block PB1 with regard to the referencesignal with the high level is x11H, as illustrated in FIG. 16B,illumination YH1 is calculated by substituting reference data x11H intothe light amount characteristic function fa1 a. Then, as illustrated inFIGS. 16A and 16B, the current value iH and the illumination YH1 of thereference signal with the high level are associated. Thus, theillumination of the pixel block PB1 equivalent to the reference signalwith the high level is estimated to be the illumination YH1.

Similarly, the illumination of the pixel block PB2 equivalent to thereference signals with the low level, the middle level, and the highlevel is estimated to be the illumination YL2, the illumination YM2, andthe illumination YH2.

In addition, similarly, the illumination of each pixel block PBequivalent to the reference signals with the low level, the middlelevel, and the high level is estimated.

Then, the reference signal characteristic estimation unit 422 generatesa reference signal characteristic table indicating the reference signalcharacteristic of each pixel block PB and causes the nonvolatile memory431 to store the reference signal characteristic table.

FIG. 17 illustrates an example of the reference signal characteristictable. In this example, the illumination of each pixel block PBequivalent to the current value iL, the current value iM, and thecurrent value iH of the reference signals with the low level, the middlelevel, and the high level is registered.

At this time, the data indicating the light amount characteristicfunction of each pixel block PB and the measurement result of thereference data used to generate the reference signal characteristictable are erased from the volatile memory 432.

Note that FIG. 16A illustrates the graphs of the reference signalcharacteristic function fb1 a indicating the reference signalcharacteristic of the pixel block PB1 and the reference signalcharacteristic function fb2 a indicating the reference signalcharacteristic of the pixel block PB2.

For example, points P21L, P21M, and P21H in FIG. 16A are pointscorresponding to reference data x21L, reference data x21M, and referencedata x21H of the pixel block PB1 with regard to the reference signals ofthe current values iL, iM, and iH. Then, the reference signalcharacteristic function fb1 a is generated through quadraticinterpolation based on the points P21L, P21M, and P21H.

Points P22L, P22M, and P22H in FIG. 16A are points corresponding toreference data x22L, reference data x22M, and reference data x22H of thepixel block PB2 with regard to the reference signals of the currentvalues iL, iM, and iH. Then, the reference signal characteristicfunction fb2 a is generated through quadratic interpolation based on thepoints P22L, P22M, and P22H.

Incidentally, as described above, the characteristic of each pixel 301of the pixel array portion 201 is rarely affected by the ambienttemperature. Accordingly, the characteristic of the added signal of eachpixel block PB output from the pixel array portion 201 is substantiallyconstant irrespective of the ambient temperature. That is, the relationbetween the illumination (the amount of incident light) of each pixelblock PB and the added signal output from each pixel block PB is notsubstantially changed in accordance with the ambient temperature.

On the other hand, the reference signal is used in online calibration tobe described below instead of the added signal. Then, as describedabove, the relation between the illumination of each pixel block PB andthe added signal output from each pixel block PB is not substantiallychanged in accordance with the ambient temperature. Accordingly, therelation between the reference signal and the illumination of each pixelblock PB is also considered to be substantially constant irrespective ofthe ambient temperature. That is, the reference signal characteristic ofeach pixel block PB can be considered to be substantially constantirrespective of the ambient temperature.

In this way, the offline calibration process is executed. Through theoffline calibration, a variation in the characteristic of each pixel 301(each pixel block PB) in which an influence of temperature is small ismainly calibrated.

<Imaging Process>

Next, an imaging process executed by the imaging device 100 will bedescribed with reference to the flowchart of FIG. 18.

For example, this process starts when the imaging device 100 is poweredon, and ends when the imaging device 100 is powered off.

In step S101, the imaging device 100 is set to the sensing mode. Themode control unit 404 supplies the mode signal indicating the setsensing mode to the input control unit 401, the correction processingunit 402, and the control unit 202.

Thus, a process of photographing a low-resolution image is started.

Specifically, the following process is started.

The pixel array portion 201 generates the added signal of each pixelblock PB and supplies the added signal to the input control unit 205.

The input control unit 205 supplies the added signal of each pixel blockPB to the analog signal processing unit 206.

The analog signal processing unit 206 executes the logarithmicconversion and the voltage conversion on the added signal of each pixelblock PB and supplies the converted added signal to the AD conversionunit 207.

The AD conversion unit 207 converts the analog added signal of eachpixel block PB into the digital added data and supplies the added dataof each pixel block PB to the input control unit 401 of the digitalsignal processing unit 209.

The input control unit 401 supplies the added data of each pixel blockPB to the correction processing unit 402.

The correction unit 412 corrects the added data of each pixel block PBon the basis of the correction function of each pixel block PB. Notethat in a case in which an online calibration process to be describedbelow has not yet been executed, for example, a default correctionfunction or a correction function used finally in a previous imagingprocess is used.

In addition, the correction unit 412 generates low-resolution image datain which the corrected data of each pixel block PB is arranged in thearray order of the pixel blocks PB. The correction unit 412 supplies thelow-resolution image data to the event detection unit 403 and theexposure control unit 405.

In step S102, the correction function generation unit 423 determineswhether or not the online calibration is executed. In a case in which anexecution condition of the online calibration is satisfied, thecorrection function generation unit 423 determines that the onlinecalibration process is executed and the process proceeds to step S103.

Note that any execution condition of the online calibration can be set.

For example, when the imaging device 100 is powered on, when the imagingdevice 100 returns from a standby state, and when the normal modetransitions to the sensing mode, the online calibration is executed.

For example, the online calibration is executed in accordance with apredetermined execution schedule. Thus, for example, the onlinecalibration is executed periodically (for example, hourly).

In addition, for example, the online calibration is executed on thebasis of the ambient temperature. For example, in a case in which theambient temperature reaches a predetermined temperature, a case in whichthe ambient temperature is changed by a predetermined threshold or morefrom the time of execution of a previous online calibration, or thelike, the online calibration is executed.

Further, for example, in a case in which an instruction to execute theonline calibration is given by the user via the manipulation unit 105,the online calibration is executed.

In step S103, the imaging device 100 executes the online calibrationprocess.

<Online Calibration Process>

Here, the details of the online calibration process will be describedwith reference to the flowchart of FIG. 19.

In step S151, the image sensor 102 is set to the online calibrationmode. The mode control unit 404 supplies a mode signal indicating theset online calibration mode to the input control unit 401, thecorrection processing unit 402, and the control unit 202. Then as in theprocess of step S9 of FIG. 14, the signal to be supplied to the analogsignal processing unit 206 is switched from the added signal to thereference signal.

In steps S152 to S157, the correction function generation unit 423executes processes similar to steps S10 to S15 of FIG. 14.

Thus, the reference data of each pixel block PB with regard to thereference signals with the levels of three stages is measured and theaverage value of the reference data is calculated.

In step S158, the correction function generation unit 423 generates thecorrection function on the basis of the reference signal characteristicand the measurement result of the reference data.

Here, an example of a method of generating the correction function willbe described with reference to FIG. 20.

FIG. 20A illustrates a graph of the same reference signal characteristicfunction fb1 a and reference signal characteristic function fb2 a asthose of FIG. 16A.

FIG. 21B illustrates a graph of a correction function fc1 a of the pixelblock PB1 and a correction function fc2 a of the pixel block PB2. Thehorizontal axis represents added data (an output value of the ADC 321) xand the vertical axis represents corrected data y.

For example, for the pixel block PB1, reference data (an average valueof the reference data) x31L with regard to the reference signal with alow level, reference data (an average value of the reference data) x31Mwith regard to the reference signal with a middle level, and referencedata (an average value of the reference data) x31H with regard to thereference signal with a high level can be obtained through the processesof steps S152 to S157. On the other hand, for the pixel block PB1,illumination YL1 equivalent to the reference signal with the low level,illumination YM1 equivalent to the reference signal with the middlelevel, and illumination YH1 equivalent to the reference signal with thehigh level are obtained based on the reference signal characteristictable stored in the nonvolatile memory 431.

Thus, the reference data x31L is associated with the illumination YL1,the reference data x31M is associated with the illumination YM1, and thereference data x31H is associated with the illumination YH1. In FIG.21B, points P31L, P31M, and P31H are points corresponding tocombinations of the reference data x31L and the illumination YL1, thereference data x31M and the illumination YM1, and the reference datax31H and the illumination YH1, respectively.

Then, a correction function fc1 a of the pixel block PB1 is generatedthrough quadratic interpolation based on the points P31L, P31M, andP31H.

In addition, for example, for the pixel block PB2, reference data (anaverage value of the reference data) x32L with regard to the referencesignal with a low level, reference data (an average value of thereference data) x32M with regard to the reference signal with a middlelevel, and reference data (an average value of the reference data) x32Hwith regard to the reference signal with a high level can be obtainedthrough the processes of steps S152 to S157. On the other hand, for thepixel block PB2, illumination YL2 equivalent to the reference signalwith the low level, illumination YM2 equivalent to the reference signalwith the middle level, and illumination YH2 equivalent to the referencesignal with the high level are obtained based on the reference signalcharacteristic table stored in the nonvolatile memory 431.

Thus, the reference data x32L is associated with the illumination YL2,the reference data x32M is associated with the illumination YM2, and thereference data x32H is associated with the illumination YH2. In FIG.21B, points P32L, P32M, and P32H are points corresponding tocombinations of the reference data x32L and the illumination YL2, thereference data x32M and the illumination YM2, and the reference datax32H and the illumination YH2, respectively.

Then, a correction function fc2 a of the pixel block PB2 is generatedthrough quadratic interpolation based on the points P32L, P32M, andP32H.

Similarly, a correction function of each pixel block PB is generated.

Then, the correction function generation unit 423 supplies data (forexample, a coefficient of the interpolation function) indicating thecorrection function of each pixel block PB to the correction unit 412.

Thereafter, the online calibration process ends. Through the onlinecalibration, a variation in the characteristic of the ADC 321 of the ADconversion unit 207 and the logarithmic conversion circuit 311 and thevoltage conversion circuit 312 of the analog signal processing unit 206in which an influence of temperature is large is mainly calibrated.

Referring back to FIG. 18, in step S104, the sensing mode is set as inthe process of step S101.

Thereafter, the process proceeds to step S105.

Conversely, in a case in which the execution condition of the onlinecalibration is not satisfied in step S102, the correction functiongeneration unit 423 determines that the online calibration process isnot executed. The processes of steps S103 and S104 are skipped and theprocess proceeds to step S105.

In step S105, the event detection unit 403 determines whether or not anevent occurs. The event detection unit 403 executes a process ofdetecting a predetermined event on the basis of the low-resolution imagedata. In a case in which the event detection unit 403 does not detectthe predetermined event, it is determined that the event does not occur.The process returns to step S102.

Thereafter, the processes of steps S102 to S105 are repeatedly executeduntil it is determined in step S105 that the event occurs. Thus,whenever the predetermined execution condition is satisfied during thesensing mode, the online calibration is executed.

Conversely, in a case in which the event detection unit 403 detects thepredetermined event in step S105, it is determined that the event occursand the process proceeds to step S106.

In step S106, the imaging device 100 executes exposure control.

Specifically, the event detection unit 403 notifies the mode controlunit 404 and the exposure control unit 405 that the event occurs.

The exposure control unit 405 sets an appropriate exposure amount on thebasis of the corrected data of each pixel of the latest low-resolutionimage data. For example, brightness of a subject and a background isdetected on the basis of the low-resolution image data and an exposureamount in accordance with the detected brightness is set. In this case,the image sensor 102, in particular, the pixel array portion 201, isused as an illumination meter for exposure control.

Note that the method of calculating the exposure amount is notparticularly limited.

The exposure control unit 405 sets an exposure time (shutter speed) anda gain (sensitivity) of the image sensor 102 and the size of thediaphragm of the optical unit 101 on the basis of the calculatedexposure amount. At this time, control amounts of the exposure time, thegain, and the size of the diaphragm are distributed so that the exposureamount reaches a target value while suppressing noise or deviation of animage.

The exposure control unit 405 supplies an exposure control signalindicating the set exposure time and gain to the control unit 202 of theimage sensor 102. Thus, the exposure time and the gain of the imagesensor 102 are set to the set values.

In addition, the exposure control unit 405 supplies the exposure controlsignal indicating the set size of the diaphragm to the optical unit 101.Thus, the size of the diaphragm of the optical unit 101 is set to theset value.

In step S107, the imaging device 100 is set to the normal mode. The modecontrol unit 404 supplies the mode signal indicating the set normal modeto the input control unit 401, the correction processing unit 402, andthe control unit 202.

Thus, a process of photographing a high-resolution image is started.Specifically, the following process is started.

The pixel array portion 201 generates a pixel signal of each pixel 301and supplies the generated pixel signal to the input control unit 205.

The input control unit 205 supplies the pixel signal of each pixel 301to the AD conversion unit 207 via the analog signal processing unit 206.

The AD conversion unit 207 converts an analog pixel signal of each pixel301 into digital pixel data and supplies the image data of each pixel301 to the input control unit 401 of the digital signal processing unit209.

The input control unit 401 supplies the pixel data of each pixel 301 tothe image processing unit 406.

The image processing unit 406 generates high-resolution image data inwhich the pixel data of each pixel 301 is arranged in the array order ofthe pixels 301. The image processing unit 406 supplies thehigh-resolution image data to the DSP circuit 103.

The DSP circuit 103 executes predetermined digital signal processing onthe high-resolution image data. The DSP circuit 103 supplies theprocessed high-resolution image data to the display unit 104 and causesan image based on the high-resolution image data to be displayed or tobe supplied and recorded on the recording unit 107.

In step S108, the mode control unit 404 determines whether or not thenormal mode ends. The determination process is repeatedly executed untilit is determined that the normal mode ends. Then, for example, in a casein which an ending condition of the normal mode is satisfied, the modecontrol unit 404 determines that the normal mode ends, and then theprocess returns to step S101.

The ending condition of the normal mode is set based on, for example, atime or an amount of recorded high-resolution image data. For example,when a time or an amount of high-resolution image data necessary foranalysis of the event is recorded, it is determined that the endingcondition of the normal mode is satisfied.

In addition, the ending condition of the normal mode is set based on,for example, whether the event occurs. For example, in a case in whichthe detected event ends, it is determined that the ending condition ofthe normal mode is satisfied. In this case, for example, the eventdetection unit 403 continues the process of detecting the event on thebasis of the high-resolution image data in the normal mode.

Further, for example, in a case in which an instruction to end thenormal mode is given by the user via the manipulation unit 105, it isdetermined that the ending condition of the normal mode is satisfied.

Thereafter, the process subsequent to step S101 is executed.

As described above, the online calibration is executed a suitable time,and the process of imaging the low-resolution image and the process ofdetecting the event are executed during the sensing mode while updatingthe correction function of each pixel block PB.

Thus, the correction function of each pixel block PB is appropriatelyadjusted in accordance with ambient temperature.

Here, an example of the calibration of the correction function will bedescribed with reference to FIG. 21.

FIG. 21A illustrates a graph of a light amount characteristic functionas in FIG. 20A. Specifically, FIG. 21A illustrates a graph of thereference signal characteristic function fb1 a of the pixel block PB1.

FIG. 21B illustrates a graph of the correction function as in FIG. 20B.Specifically, FIG. 21B illustrates a graph of the correction function ofthe pixel block PB1. The correction function fc1 a is the same as thecorrection function fc1 a of FIG. 20B and is, for example, a correctionfunction of the pixel block PB1 in a case in which the ambienttemperature is 27 degrees.

For example, in a case in which the ambient temperature is changed from27 degrees to 0 degrees, the reference data measured with regard to thereference signal with each level is changed from the reference datax31L, the reference data x31M, and the reference data 31H to referencedata x31L′, reference data x31M′, and reference data 31H′, respectively.In this case, a correction function fc1 a′ is generated on the basis ofa point P31L′ corresponding to the combination of the reference datax31L′ and the illumination YL1, a point P31M′ corresponding to thecombination of the reference data x31M′ and the illumination YM1, and apoint P31L′ corresponding to the combination of the reference data x31H′and the illumination YH1.

In addition, for example, in a case in which the ambient temperature ischanged from 27 degrees to 40 degrees, the reference data measured withregard to the reference signal with each level is changed from thereference data x31L, the reference data x31M, and the reference data 31Hto reference data x31L″, reference data x31M″, and reference data 31H″,respectively. In this case, a correction function fc1 a″ is generated onthe basis of a point P31L″ corresponding to the combination of thereference data x31L″ and the illumination YL1, a point P31M″corresponding to the combination of the reference data x31M″ and theillumination YM1, and a point P31L″ corresponding to the combination ofthe reference data x31H″ and the illumination YH1.

In this way, the correction function of each pixel block PB isappropriately adjusted in accordance with the ambient temperature, andthus even when the light amount characteristic of each pixel block PB ischanged in accordance with the ambient temperature, the corrected dataobtained with regard to the same amount of incident light issubstantially constant. In addition, a variation in the light amountcharacteristic between the pixel blocks PB is corrected and thecorrected data obtained with regard to the same amount of incident lightis substantially constant between the pixel blocks PB.

Thus, the quality of the low-resolution image data is improved anddetection precision of the event is improved.

In addition, the exposure of the imaging device 100 can be appropriatelyadjusted before the sensing mode transitions to the normal mode, andthus the high-resolution image data can be acquired from the first framethrough the appropriate exposure.

Specifically, the lower drawing of FIG. 22 is a graph indicating anexample of a change in the corrected data y with respect to the sameamount of incident light as that during the sensing mode, as in thelower drawing of FIG. 13.

In this way, the online calibration is executed appropriately during thesensing mode, so that the value of the corrected data y with regard tothe same amount of incident light is maintained to be substantiallyconstant. Thus, a difference Δy between the value of the corrected datay at time t0 at the time of starting of the sensing mode and thecorrected data y at time t1 at the time of ending of the sensing modebecomes very small.

Accordingly, in a case in which the automatic exposure control isexecuted on the basis of the corrected data y, the exposure controlamount of the imaging device 100 is substantially the same at time t0and time t1 even when the amount of incident light (brightness of asubject or a background) is the same.

Accordingly, before the sensing mode transitions to the normal mode, theexposure control of the imaging device 100 is appropriately executed.

In addition, by repeating the online calibration during the sensingmode, the exposure control can be executed quickly and appropriatelyeven when the online calibration is not executed at the time of thetransition to the normal mode.

Further, the added signal has a broad dynamic range since the addedsignal is a logarithmically converted signal. Accordingly, by executingthe exposure control on the basis of the corrected data generated on thebasis of the added signal, it is possible to broaden a range of thebrightness with which the exposure control can be executed with highprecision.

In addition, in the sensing mode, a processing load or power consumptionis reduced by performing the process of detecting the event on the basisof the low-resolution image data with a small number of pixels.

Further, since the reference signal is used in the online calibration,it is not necessary to provide a thermometer or a light source forcalibration in the imaging device 100. Accordingly, it is possible toachieve miniaturization, low cost, and power saving of the imagingdevice 100. In addition, since no light source is used, onlinecalibration can be executed with high precision without an influence ofdisturbance light.

2. Second Embodiment

Next, a second embodiment of the present technology will be describedwith reference to FIG. 23.

The second embodiment is an embodiment in a case in which a referencesignal of each pixel block PB is proportional to an amount of incidentlight.

FIG. 23 illustrates a relation between light amount characteristicfunctions and reference signal characteristic functions as in FIG. 16.FIG. 23A illustrates a graph of a reference signal characteristicfunction fb1 b of the pixel block PB1 and a reference signalcharacteristic function fb2 b of the pixel block PB2. The referencesignal characteristic functions fb1 b and fb2 b are proportionalfunctions. That is, in the pixel blocks PB1 and PB2, a reference signalis proportional to illumination (amount of incident light).

FIG. 23B illustrates a graph of the same light amount characteristicfunction fa1 a and light amount characteristic function fa2 a as thoseof FIG. 16B.

In this case, for example, when illumination with regard to a referencesignal with a certain level is known, illumination with regard toreference signals with other levels can be obtained by calculation.

For example, in a case in which reference data (an average value of thereference data) x11M of the pixel block PB1 with regard to the referencesignal with the middle level is measured, the illumination YM1 of thepixel block PB1 equivalent to the reference signal with the middle levelis obtained by substituting the reference data x11M into the lightamount characteristic function fa1 a.

Then, the reference signal characteristic function fb1 b is estimated onthe basis of a point P51M corresponding to a combination of theillumination YM1 and the current value iM of the reference signal withthe middle level. In addition, since the current value iL of thereference signal with the low level is known, the illumination YL1 ofthe pixel block PB1 equivalent to the reference signal with the lowlevel is calculated by substituting the current value iL into thereference signal characteristic function fb1 b. Similarly, since thecurrent value iH of the reference signal with the high level is known,the illumination YH1 of the pixel block PB1 equivalent to the referencesignal with the high level is calculated by substituting the currentvalue iH into the reference signal characteristic function fb1 b.

Accordingly, by merely measuring the reference data x11M with regard tothe reference signal with the middle level, it is possible to estimatethe illumination of the pixel block PB11 equivalent to the referencesignal with each level. In addition, only the illumination YM1corresponding to the reference signal with the middle level may berecorded in the reference signal characteristic table, and it is notnecessary to record the illumination YL1 and the illumination YH1corresponding to the reference signals with the low level and the highlevel.

Similarly, even for the pixel block PB12, by merely measuring referencedata x12M with regard to a reference signal with the middle level, it ispossible to estimate the illumination of the pixel block PB12 equivalentto the reference signal with each level. In addition, only theillumination YM2 corresponding to the reference signal with the middlelevel may be recorded in the reference signal characteristic table, andit is not necessary to record the illumination YL2 and the illuminationYH2 corresponding to the reference signals with the low level and thehigh level.

In this way, in a case in which the amount of incident light of eachpixel block PB and the reference signal have a proportional relation,the online calibration process, in particular, the process of estimatingthe reference signal characteristic of each pixel block PB, islightened. In addition, the amount of data of the reference signalcharacteristic table can be reduced. As a result, it is possible toreduce the capacity of the nonvolatile memory 431.

3. Third Embodiment

Next, a third embodiment of the present technology will be describedwith reference to FIGS. 24 to 26.

The third embodiment is an embodiment in a case in which not only is thereference signal of each pixel block PB proportional to an amount ofincident light, as in the second embodiment, but the light amountcharacteristic of each pixel block PB has linearity. Here, the fact thatthe light amount characteristic of each pixel block PB has the linearitymeans that added data substantially linearly changes with respect to theamount of incident light in each pixel block PB.

First, an example of a method of estimating a light amountcharacteristic of each pixel block PB will be described with referenceto FIG. 24.

FIG. 24 illustrates a graph of light amount characteristic functions asin FIG. 15. Specifically, FIG. 24 illustrates a graph of light amountcharacteristic functions fa1 b to fa3 b of the pixel blocks PB1 to PB3.The light amount characteristic functions fa1 b to fa3 b are expressedas linear functions that have linearity.

In this case, for example, a light reception surface of the pixel arrayportion 201 of the image sensor 102 is set substantially uniformly withillumination YR with a predetermined level by using an external lightsource. In this state, added data (an average value of the added data)of each pixel block PB is measured as in the above-described process.For example, added data (an average value of the added data) x101R ofthe pixel block PB1, added data (an average value of the added data)x102R of the pixel block PB2, and added data (an average value of theadded data) x103R of the pixel block PB3 at the time of illumination TRare measured.

Subsequently, for example, a signal to be supplied to the analog signalprocessing unit 206 is switched from an added signal to a referencesignal and the output of the reference signal is stopped. Thus, theinput of the signal to the analog signal processing unit 206 is stoppedand the added data in a state in which the illumination in the pixelblock PB is virtually set to 0 is output from the AD conversion unit207.

In this state, as in the above-described process, the added data (anaverage value of the added data) of each pixel block PB is measured. Forexample, added data (an average value of the added data) x101D of thepixel block PB1, added data (an average value of the added data) x102Dof the pixel block PB2, and added data (an average value of the addeddata) x103D of the pixel block PB3 at the time of illumination of 0 aremeasured.

Then, the light amount characteristic function of each pixel block PB isestimated on the basis of the added data (the average value of the addeddata) at the time of the illumination of 0 and the added data (theaverage value of the added data) at the time of the illumination YR.

For example, the light amount characteristic function fa1 b of the pixelblock PB1 is estimated through linear interpolation based on a pointP101D corresponding to the added data x101D at the time of theillumination of 0 and a point P101R corresponding to the added datax101R at the time of the illumination YR. The light amountcharacteristic function fa2 b of the pixel block PB2 is estimatedthrough linear interpolation based on a point P102D corresponding to theadded data x102D at the time of the illumination of 0 and a point P102Rcorresponding to the added data x102R at the time of the illuminationYR. The light amount characteristic function fa3 b of the pixel blockPB3 is estimated through linear interpolation based on a point P103Dcorresponding to the added data x103D at the time of the illumination of0 and a point P103R corresponding to the added data x103R at the time ofthe illumination YR.

The light amount characteristic functions of the other pixel blocks PBare also estimated in accordance with a similar method.

Then, data (for example, a coefficient of the light amountcharacteristic function) indicating the light amount characteristicfunction of each pixel block PB is retained in the volatile memory 432.

In this way, compared to the first embodiment, the number of times theadded data is measured is reduced and the process of estimating thelight amount characteristic of each pixel block PB is lightened.

Next, an example of a method of estimating the reference signalcharacteristic of each pixel block PB will be described with referenceto FIG. 25.

FIG. 25 illustrates relations between light amount characteristicfunctions and reference signal characteristic functions as in FIG. 16.

FIG. 25A illustrates a graph of reference signal characteristicfunctions fb1 b to fb3 b of the pixel blocks PB1 to PB2. The referencesignal characteristic functions fb1 b to f3 b are proportionalfunctions. That is, in the pixel blocks PB1 to PB3, reference signalsare proportional to illumination (amount of incident light).

FIG. 25B illustrates a graph of the same light amount characteristicfunctions fa1 b to fa3 b of the pixel blocks PB1 to PB3 as those of FIG.24.

For example, reference data (an average value of the reference data) ofeach pixel block PB with regard to the reference signal with apredetermined level (current value iR) is first measured. For example,reference data (an average value of the reference data) x111R of thepixel block PB1, reference data (an average value of the reference data)x112R of the pixel block PB2, and reference data (an average value ofthe reference data) x113R of the pixel block PB3 with regard to thereference signal with a predetermined level are measured.

Subsequently, illumination YR1 of the pixel block PB1 equivalent to thereference signal with the predetermined level is estimated bysubstituting the reference data x111R into the light amountcharacteristic function fa1 b. Illumination YR2 of the pixel block PB2equivalent to the reference signal with the predetermined level isestimated by substituting the reference data x112R into the light amountcharacteristic function fa2 b. Illumination YR3 of the pixel block PB3equivalent to the reference signal with the predetermined level isestimated by substituting the reference data x113R into the light amountcharacteristic function fa3 b.

Illumination equivalent to the reference signal with the predeterminedlevel of the other pixel blocks PB is designated in accordance with asimilar method.

Then, the illumination equivalent to the reference signal with thepredetermined level in each pixel block PB is recorded in the referencesignal characteristic table. In addition, the reference signalcharacteristic function of each pixel block PB is estimated inaccordance with a method similar to that of the second embodiment.

In this way, compared to the first embodiment, the number of times thereference data is measured is reduced and the process of estimating thereference signal characteristic of each pixel block PB is lightened. Inaddition, the amount of data of the reference signal characteristictable is reduced.

Next, an example of a method of generating the correction function ofeach pixel block PB will be described with reference to FIG. 26.

FIG. 26 illustrates relations between reference signal characteristicfunctions and correction functions as in FIG. 20.

FIG. 26A illustrates a graph of the same reference signal characteristicfunctions fb1 a to fb1 c as those of FIG. 25A.

FIG. 26B illustrates a graph of the correction functions as in FIG. 20B.Specifically, FIG. 26B illustrates a graph of correction functions fc1 bto fc3 b of the pixel blocks PB1 to PB3.

For example, reference data (an average value of the reference data) ofeach pixel block PB with regard to the reference signal with the samelevel as at the time of estimating the reference signal characteristic(current value iR) is measured. For example, reference data (an averagevalue of the reference data) x131R of the pixel block PB1, referencedata (an average value of the reference data) x132R of the pixel blockPB2, and reference data (an average value of the reference data) x133Rof the pixel block PB3 with regard to the reference signal with apredetermined level are measured.

Subsequently, the output of the reference signals is stopped. Thus, theinput of the signal to the analog signal processing unit 206 is stoppedand the reference data in the state in which the current value of thereference signal is set to 0 (at the time of a reference signal of 0) isoutput from the AD conversion unit 207.

In this state, reference data (an average value of the reference data)of each pixel block PB is measured. For example, reference data (anaverage value of the reference data) x131D of the pixel block PB1,reference data (an average value of the reference data) x132D of thepixel block PB2, and reference data (an average value of the referencedata) x133D of the pixel block PB3 at the time of the reference signalof 0 are measured.

Then, the correction function of each pixel block PB is generated on thebasis of the reference data at the time of the reference signal of 0 andthe measurement result of the reference data with regard to thereference signal with a predetermined level.

For example, the correction function fc1 b of the pixel block PB1 isgenerated through linear interpolation based on a point P131Dcorresponding to the reference data x131D at the time of the referencesignal of 0 and a point P131R corresponding to the reference data x131Rwith regard to the reference signal with the predetermined level. Thecorrection function fc2 b of the pixel block PB2 is generated throughlinear interpolation based on a point P132D corresponding to thereference data x132D at the time of the reference signal of 0 and apoint P132R corresponding to the reference data x132R with regard to thereference signal with the predetermined level. The correction functionfc3 b of the pixel block PB3 is generated through linear interpolationbased on a point P133D corresponding to the reference data x133D at thetime of the reference signal of 0 and a point P133R corresponding to thereference data x133R with regard to the reference signal with thepredetermined level.

For other pixel blocks PB, correction functions are also generated inaccordance with a similar method.

In this way, compared to the first embodiment, the number of times thereference data is measured is reduced and the process of generating thecorrection function of each pixel block PB is lightened.

Note that the correction function in this case is expressed in the formof y=b1×x+b0. The coefficient b1 is a coefficient for correcting avariation in a gain of a circuit generating the added data of the pixelblock PB. The coefficient b0 is a coefficient for correcting a variationin an offset characteristic of the circuit generating the added data ofthe pixel block PB.

4. Fourth Embodiment

Next, a fourth embodiment of the present technology will be describedwith reference to FIG. 27.

The fourth embodiment is an embodiment in a case in which the lightamount characteristic of each pixel block PB complicatedly changes, itis difficult to approximate the light amount characteristic function toa linear function or a quadratic function, and it is consequentlydifficult to express the correction function as a linear function or aquadratic function. Note that in the reference signal characteristic ofeach pixel block PB, the reference signal and the illumination (theamount of incident light) have a proportional relation as in the secondand third embodiments.

FIG. 27 illustrates a relation of a reference signal characteristicfunction and a correction function as in FIG. 20.

FIG. 27A illustrates a graph of a reference signal characteristicfunction fb1 d of the pixel block PB1. The reference signalcharacteristic function fb1 d is a proportional function. That is, theillumination of the pixel block PB1 and the reference signal have aproportional relation.

FIG. 27B is a graph of a correction function fc1 c of the pixel blockPB1. The correction function fc1 c complicatedly changes, and thus it isdifficult to express the correction function fc1 c as a linear functionor a quadratic function.

In this case, reference data (an average value of the reference data)x161A to reference data (an average value of the reference data) x161Eof the pixel block PB1 with respect to reference signals with levels Ato E (current values iA to iE) are measured.

In addition, for example, illumination YC1 of the pixel block PB1equivalent to the reference signal with the level C (the current valueiC) is recorded in the reference signal characteristic table. Then, afunction connecting the origin to a point P151C corresponding to acombination of the current value iC and the illumination YC1 isestimated as the reference signal characteristic function fb1 d of thepixel block PB1.

Further, illumination YA1, illumination YB1, illumination YD1, andillumination YE1 equivalent to the reference signals with the level A,the level B, the level D, and the level E are calculated by substitutingthe current value iA, the current value iB, the current value iD, andthe current iE into the reference signal characteristic function fb1 d.Thus, the reference data x161A is associated with the illumination YA1,the reference data x161B is associated with the illumination YB1, thereference data x161D is associated with the illumination YD1, and thereference data x161E is associated with the illumination YE1.

Subsequently, the range of the added data is divided into a range from 0to the reference data x161A, a range from the reference data x161A tothe reference data x161B, a range from the reference data x161B to thereference data x161C, a range from the reference data x161C to thereference data x161D, and a range from the reference data x161D to thereference data x161E.

Subsequently, the linear function or the quadratic function indicatingthe relation between the added data x and the corrected data y in eachrange is generated for each of the divided ranges through linearinterpolation or quadratic interpolation. Accordingly, in this case, theinterpolation function of the pixel block PB1 is represented bycombining the functions for each range of the added data.

The correction functions of the other pixel blocks PB are also generatedin accordance with a similar method.

Then, the data (for example, coefficients of each function included inthe interpolation function) indicating the interpolation function ofeach pixel block PB is retained in the volatile memory 432.

Thus, even in a case in which the light amount characteristic of thepixel block PB complicatedly changes, the interpolation function can beappropriately generated.

5. Modification Examples

Hereinafter, modification examples of the embodiments of theabove-described present technology will be described.

In the foregoing description, the case in which the calibration of thecorrection function of the added data based on the added signal obtainedby adding the pixel signals in units of the pixel blocks PB is executedhas been exemplified. However, for example, the present technology canalso be applied to a case in which the calibration of the correctionfunction of the pixel data based on the pixel signal of each pixel isexecuted. For example, the online calibration can be executed using thereference signal and the correction function of each pixel can beadjusted.

In addition, in the foregoing description, the case in which thecalibration of the correction function used to correct thelogarithmically converted added data is executed has been exemplified.However, the present technology can also be applied to a case in which acorrection function of added data or pixel data which is notlogarithmically converted is calibrated.

Further, in the foregoing description, the example in which the pixelsignals (the added signals) output from the pixel array portion 201 arereduced by adding the pixel signals of the plurality of pixels 301during the sensing mode has been described, but the pixel signals may bereduced in accordance with another method. For example, the pixelsignals output from the pixel array portion 201 may be reduced bydecimating the pixels outputting the pixel signals, that is, by reducingthe pixels outputting the pixel signals. In addition, for example, thepixel signals output from the pixel array portion 201 may be reduced bylengthening an output interval of the pixel signals.

In addition, the present technology can also be applied to, for example,a case in which the online calibration is executed during a normalimaging process.

Further, in particular, a kind, configuration, or the like of the imagesensor is not limited as long as the image sensor to which the presenttechnology can be applied is an image sensor performing the onlinecalibration. For example, the case in which the pixel 301 has the sharedpixel structure has been exemplified above, but the present technologycan also be applied to an image sensor not including a pixel that hasthe shared pixel structure.

In addition, for example, the present technology can also be applied toa temperature detection element 501 in FIG. 28. Note that in thetemperature detection element 501 in FIG. 28, the same reference singsare given to portions corresponding to the image sensor 102 in FIG. 2and the description thereof will be appropriately omitted.

The temperature detection element 501 is different from the image sensor102 in that a thermal array portion 511 is provided instead of the pixelarray portion 201.

In the thermal array portion 511, for example, a thermal sensor thatconverts temperature into an electric signal is provided in each pixelinstead of the photoelectric conversion element.

Then, in the temperature detection element 501, for example, offlinecalibration can be executed using an infrared (IR) light source inaccordance with a method similar to that of the image sensor 102. Inaddition, in the temperature detection element 501, for example, theonline calibration can be executed using a reference signal in a methodsimilar to that of the image sensor 102.

Thus, it is possible to correct a variation in a detectioncharacteristic of temperature for each pixel and realize a highlyprecise temperature meter function.

In addition, in the first embodiment, the example in which the lightamount characteristic is estimated on the basis of the measurementresult of the added data with respect to the illumination at the threestages has been described. However, for example, the light amountcharacteristic may be estimated on the basis of a measurement result ofadded data with respect to illumination at four or more stages. Inaddition, in the first embodiment, the example in which the correctionfunction is generated on the basis of the measurement result of thereference data with respect to the reference signal at the three stageshas been described. However, for example, a correction function may begenerated on the basis of a measurement result of reference data withrespect to reference signals at four or more levels.

6. Application Example of Present Technology

For example, the present technology can be applied to various cases inwhich light such as visible light, infrared light, ultraviolet light, oran X ray is sensed, as illustrated in FIG. 29.

-   -   Devices that take images used for viewing, such as a digital        camera and a portable appliance with a camera function.    -   Devices used for traffic, such as an in-vehicle sensor that        takes images of the front and the back of a car, surroundings,        the inside of the car, and the like, a monitoring camera that        monitors travelling vehicles and roads, and a distance sensor        that measures distances between vehicles and the like, which are        used for safe driving (e.g., automatic stop), recognition of the        condition of a driver, and the like.    -   Devices used for home electrical appliances, such as a TV, a        refrigerator, and an air conditioner, to takes images of a        gesture of a user and perform appliance operation in accordance        with the gesture.    -   Devices used for medical care and health care, such as an        endoscope and a device that performs angiography by reception of        infrared light.    -   Devices used for security, such as a monitoring camera for crime        prevention and a camera for personal authentication.    -   Devices used for beauty care, such as skin measurement equipment        that takes images of the skin and a microscope that takes images        of the scalp.    -   Devices used for sports, such as an action camera and a wearable        camera for sports and the like.    -   Devices used for agriculture, such as a camera for monitoring        the condition of the field.

Hereinafter, a more detailed application example will be explained.

<Example of Application to Mobile Objects>

The technology (present technology) according to an embodiment of thepresent disclosure is applicable to a variety of products. For example,the technology according to an embodiment of the present disclosure isimplemented as devices mounted on any type of mobile objects such asautomobiles, electric vehicles, hybrid electric vehicles, motorcycles,bicycles, personal mobilities, airplanes, drones, ships, and robots.

FIG. 30 is a block diagram illustrating a schematic configurationexample of a vehicle control system which is an example of a mobileobject control system to which a technology according to an embodimentof the present technology is applicable.

A vehicle control system 12000 includes a plurality of electroniccontrol units connected via a communication network 12001. In theexample illustrated in FIG. 30, the vehicle control system 12000includes a drive line control unit 12010, a body system control unit12020, a vehicle outside information detection unit 12030, a vehicleinside information detection unit 12040, and an integrated control unit12050. In addition, as functional configurations of the integratedcontrol unit 12050, a microcomputer 12051, an audio and image outputunit 12052, an in-vehicle network interface (I/F) 12053.

The drive line control unit 12010 controls the operation of devicesrelated to the drive line of the vehicle in accordance with a variety ofprograms. For example, the drive line control unit 12010 functions as acontrol device for a driving force generating device such as an internalcombustion engine or a driving motor that generates the driving force ofthe vehicle, a driving force transferring mechanism that transfers thedriving force to wheels, a steering mechanism that adjusts the steeringangle of the vehicle, a braking device that generates the braking forceof the vehicle, and the like.

The body system control unit 12020 controls the operations of a varietyof devices attached to the vehicle body in accordance with a variety ofprograms. For example, the body system control unit 12020 functions as acontrol device for a keyless entry system, a smart key system, a powerwindow device, or a variety of lights such as a headlight, a backuplight, a brake light, a blinker, or a fog lamp. In this case, the bodysystem control unit 12020 can receive radio waves transmitted from aportable device that serves instead of the key or signals of a varietyof switches. The body system control unit 12020 receives these radiowaves or signals, and controls the vehicle door lock device, the powerwindow device, the lights, or the like.

The vehicle outside information detection unit 12030 detects informationregarding the outside of a vehicle on which the vehicle control system12000 is mounted. For example, an imaging unit 12031 is connected to thevehicle outside information detection unit 12030. The vehicle outsideinformation detection unit 12030 causes the imaging unit 12031 tocapture an image outside of the vehicle and receives the captured image.The vehicle outside information detection unit 12030 may perform anobject detection process or a distance detection process for a person, avehicle, an obstacle, a sign, letters on a road, or the like on thebasis of the received image.

The imaging unit 12031 is a light sensor that receives light and outputsan electric signal in accordance with the amount of received light. Theimaging unit 12031 can output the electric signal as an image ordistance measurement information. In addition, the light received by theimaging unit 12031 may be the visible light or may be non-visible lightsuch as infrared light.

The vehicle inside information detecting unit 12040 detects informationregarding the inside of the vehicle. The vehicle inside informationdetecting unit 12040 is connected, for example, to a driver statedetecting unit 12041 that detects the state of the driver. The driverstate detecting unit 12041 may include, for example, a camera thatimages the driver. The vehicle inside information detecting unit 12040may compute the degree of the driver's tiredness or the degree of thedriver's concentration or determine whether the driver have a doze, onthe basis of detection information input from the driver state detectingunit 12041.

For example, the microcomputer 12051 can calculate a control targetvalue of the driving force generating device, the steering mechanism, orthe braking device on the basis of information acquired by the vehicleoutside information detecting unit 12030 or the vehicle insideinformation detecting unit 12040 on the inside and outside of thevehicle, and output a control instruction to the drive line control unit12010. For example, the microcomputer 12051 may perform cooperativecontrol for the purpose of executing the functions of an advanced driverassistance system (ADAS) including vehicle collision avoidance or impactreduction, follow-up driving based on the inter-vehicle distance,constant vehicle speed driving, vehicle collision warning, vehicle lanedeparture warning, or the like.

Further, the microcomputer 12051 can control the driving forcegenerating device, the steering mechanism, the braking device, or thelike on the basis of information acquired by the vehicle outsideinformation detecting unit 12030 or the vehicle inside informationdetecting unit 12040 on the areas around the vehicle, thereby performingcooperative control for the purpose of automatic driving or the likethat allows the vehicle to autonomously travel irrespective of anyoperation of a driver.

In addition, the microcomputer 12051 can output a control instruction tothe body system control unit 12020 on the basis of the informationregarding the outside of the vehicle acquired by the vehicle outsideinformation detection unit 12030. For example, the microcomputer 12051can control a head lamp in accordance with the position of a precedingvehicle or an oncoming vehicle detected by the vehicle outsideinformation detection unit 12030 and can perform cooperative control forthe purpose of anti-glaring such as switching a high beam to a low beam.

The audio and image output unit 12052 transmits an output signal of atleast one of a sound and an image to an output device capable ofvisually or aurally notifying a passenger of the vehicle or the outsideof the vehicle of information. In the example of FIG. 30, an audiospeaker 12061, a display unit 12062, and an instrument panel 12063 areexemplified as the output device. For example, the display unit 12062may include at least one of an onboard display and a head-up display.

The figure is a diagram illustrating an example of an installationposition of the imaging unit 12031.

In FIG. 31, the vehicle 12100 includes imaging units 12101, 12102,12103, 12104, and 12105 as the imaging unit 12031.

Imaging units 12101, 12102, 12103, 12104, and 12105 are positioned, forexample, at the front nose, a side mirror, the rear bumper, the backdoor, and the upper part of the windshield in the vehicle compartment ofa vehicle 12100. The imaging unit 12101 attached to the front nose andthe imaging unit 12105 attached to the upper part of the windshield inthe vehicle compartment chiefly acquire images of the area ahead of thevehicle 12100. The imaging units 12102 and 12103 attached to the sidemirrors chiefly acquire images of the areas on the sides of the vehicle12100. The imaging unit 12104 attached to the rear bumper or the backdoor chiefly acquires images of the area behind the vehicle 12100. Afront image acquired by the imaging units 12101 and 12105 is usedchiefly to detect a preceding vehicle, a pedestrian, an obstacle, atraffic light, a traffic sign, a lane, or the like.

Additionally, FIG. 31 illustrates an example of the imaging ranges ofthe imaging units 12101 to 12104. An imaging range 12111 represents theimaging range of the imaging unit 12101 attached to the front nose.Imaging ranges 12112 and 12113 respectively represent the imaging rangesof the imaging units 12102 and 12103 attached to the side mirrors. Animaging range 12114 represents the imaging range of the imaging unit12104 attached to the rear bumper or the back door. For example,overlaying image data captured by the imaging units 12101 to 12104offers an overhead image that looks down on the vehicle 12100.

At least one of the imaging units 12101 to 12104 may have a function ofacquiring distance information. For example, at least one of the imagingunits 12101 to 12104 may be a stereo camera including a plurality ofimage sensors or may be an image sensor that includes pixels for phasedifference detection.

For example, the microcomputer 12051 can extract a 3-dimensional objecttraveling at a predetermined speed (for example, 0 or more km/h) insubstantially the same direction as the vehicle 12100 as a precedingvehicle by particularly using a closest 3-dimensional object on a travelroad of the vehicle 12100 by obtaining a distance to each 3-dimensionalobject within the imaging ranges 12111 to 12114 and a temporal change inthe distance (a relative speed to the vehicle 12100) on the basis ofdistance information obtained from the imaging units 12101 to 12104.Further, the microcomputer 12051 can set an inter-vehicle distance to beensured in advance before a preceding vehicle and perform automaticbrake control (also including follow-up stop control) or automaticacceleration control (also including follow-up oscillation control). Inthis way, it is possible to perform cooperative control for the purposeof automatic driving or the like that allows the vehicle to autonomouslytravel irrespective of any operation of a driver.

For example, the microcomputer 12051 can classify and extract3-dimensional object data regarding 3-dimensional objects as other3-dimensional objects such as motorcycles, normal vehicles, largevehicles, pedestrians, and electric poles on the basis of the distanceinformation obtained from the imaging units 12101 to 12104 and can usethe other 3-dimensional objects to automatically avoid obstacles. Forexample, the microcomputer 12051 identifies obstacles around the vehicle12100 as obstacles which can be viewed by a driver of the vehicle 12100and obstacles which are difficult to view. Then, the microcomputer 12051can determine a collision risk indicating a danger of collision witheach obstacle and output a warning to the driver via the audio speaker12061 or the display unit 12062 in a situation in which there is acollision possibility since the collision risk is set to be equal to orgreater than a set value or can perform driving assistance for collisionavoidance by performing forced deceleration or avoidance steering iv viathe drive line control unit 12010.

At least one of the imaging units 12101 to 12104 may be an infraredcamera that detects infrared light. For example, the microcomputer 12051can recognize a pedestrian by determining whether or not there is thepedestrian in captured images of the imaging units 12101 to 12104. Thepedestrian can be recognized, for example, in a procedure in whichfeature points are extracted in the captured images of the imaging units12101 to 12104 serving as infrared cameras and a procedure in which aseries of feature points indicating a contour of an object are subjectedto a pattern matching process to determine whether or not there is thepedestrian. The microcomputer 12051 determines that there is thepedestrian in the captured images of the imaging units 12101 to 12104.When the pedestrian is recognized, the audio and image output unit 12052controls the display unit 12062 such that a rectangular contour line foremphasis is superimposed to be displayed on the recognized pedestrian.In addition, the audio and image output unit 12052 controls the displayunit 12062 such that an icon or the like indicating the pedestrian isdisplayed at a desired position.

An example of the vehicle control system to which the technologyaccording to the present disclosure can be applied has been describedabove. The technology according to the present disclosure can be appliedto the imaging unit 12031 or the like within the above-describedconfiguration. Specifically, for example, the described-above imagesensor 102 can be applied to the imaging unit 12031. Thus, whenmonitoring processing is performed during parking on the basis of animage pictured by the imaging unit 12031 in order to prevent a thief ofa vehicle, or the like, for example, it is possible to achieve powersaving by switching the sensing mode and the normal mode. In addition,when an event is detected on the areas around the vehicle, it ispossible to image a high-resolution image from the first frame throughthe appropriate exposure.

Moreover, embodiments of the present technology are not limited to theabove-described embodiments, but various changes can be made within thescope of the present technology without departing from the gist of thepresent technology.

<Combination Examples of Configurations>

Additionally, the present technology may also be configured as below.

(1)

An image sensor including:

a pixel array portion in which a plurality of pixels are disposed andwhich generates a pixel signal;

a reference signal generation unit configured to generate a referencesignal for calibration;

an analog digital (AD) conversion unit configured to execute ADconversion on the pixel signal and the reference signal to generatepixel data and reference data; and

a correction processing unit configured to correct the pixel data on abasis of the reference data.

(2)

The image sensor according to (1), in which the correction processingunit includes

a correction function generation unit configured to generate acorrection function on the basis of the reference data, and

a correction unit configured to correct the pixel data on a basis of thecorrection function.

(3)

The image sensor according to (2), in which the correction functiongeneration unit generates the correction function on a basis of areference signal characteristic indicating a correspondent relationbetween an amount of incident light of the pixels and the referencesignal and a measurement result of the reference data with regard to thereference signal with a predetermined level.

(4)

The image sensor according to (3), in which the correction processingunit further includes

a reference signal characteristic estimation unit configured to estimatethe reference signal characteristic on a basis of a light amountcharacteristic indicating a relation between the amount of incidentlight of the pixels and the pixel data and the measurement result of thereference data with regard to the reference signal with thepredetermined level.

(5)

The image sensor according to (4), in which the correction processingunit further includes

a light amount characteristic estimation unit configured to estimate thelight amount characteristic on a basis of a measurement result of thepixel data with respect to a predetermined amount of incident light.

(6)

The image sensor according to any one of (2) to (5), in which thecorrection function generation unit updates the correction function onthe basis of the reference data whenever a predetermined condition issatisfied.

(7)

The image sensor according to any one of (2) to (6), in which thecorrection function is a function of correcting a variation, between thepixels, in a light amount characteristic indicating a relation betweenan amount of incident light of the pixels and the pixel data.

(8)

The image sensor according to (1), further including:

an analog signal processing unit configured to execute analog signalprocessing on the pixel signal and the reference signal,

in which the AD conversion unit executes the AD conversion on the pixelsignal and the reference signal subjected to the analog signalprocessing.

(9)

The image sensor according to (8),

in which the analog signal processing includes a logarithmic conversionprocess for the pixel signal and the reference signal, and

the correction processing unit includes

-   -   a correction function generation unit configured to generate a        correction function of correcting a relation between an amount        of incident light of the pixels and the pixel data to a linear        relation on the basis of the reference data, and    -   a correction unit configured to correct the pixel data on a        basis of the correction function.        (10)

The image sensor according to (1), further including:

a mode control unit configured to set a first driving mode and a seconddriving mode in which the pixel signal output from the pixel arrayportion is reduced more than in the first driving mode,

in which the correction processing unit corrects the pixel data on thebasis of the reference data during the second driving mode.

(11)

The image sensor according to (10), in which the correction processingunit includes

a correction function generation unit configured to generate acorrection function on the basis of the reference data during the seconddriving mode, and

a correction unit configured to correct the pixel data on a basis of thecorrection function during the second driving mode.

(12)

The image sensor according to (11), in which the correction functiongeneration unit updates the correction function on the basis of thereference data whenever a predetermined condition is satisfied duringthe second driving mode.

(13)

The image sensor according to any one of (10) to (12), furtherincluding:

an exposure control unit configured to control exposure in the firstdriving mode on a basis of the corrected pixel data.

(14)

The image sensor according to (13), in which the exposure control unitcontrols the exposure before the second driving mode transitions to thefirst driving mode.

(15)

The image sensor according to any one of (10) to (14), in which the modecontrol unit sets the first driving mode in a case in which apredetermined event is detected on a basis of the corrected pixel datain the second driving mode.

(16)

The image sensor according to any one of (10) to (15), in which thepixel signal obtained by adding signals from a plurality of the pixelsis generated in the second driving mode.

(17)

The image sensor according to any one of (10) to (15), in which thepixels outputting the pixel signal are reduced in the second drivingmode.

(18)

A signal processing device including:

a correction processing unit configured to correct pixel data obtainedwhen an analog digital (AD) conversion unit executes AD conversion on apixel signal generated by a pixel array portion in which a plurality ofpixels are disposed, on a basis of reference data obtained when the ADconversion unit executes the AD conversion on a reference signal forcalibration.

(19)

A signal processing method including:

correcting pixel data obtained when an analog digital (AD) conversionunit executes AD conversion on a pixel signal generated by a pixel arrayportion in which a plurality of pixels are disposed, on a basis ofreference data obtained when the AD conversion unit executes the ADconversion on a reference signal for calibration.

(20)

An electronic device including:

an image sensor; and a signal processing unit configured to process asignal output from the image sensor,

in which the image sensor includes

-   -   a pixel array portion in which a plurality of pixels are        disposed and which generates a pixel signal,    -   a reference signal generation unit configured to generate a        reference signal for calibration,    -   an analog digital (AD) conversion unit configured to execute AD        conversion on the pixel signal and the reference signal to        generate pixel data and reference data, and    -   a correction processing unit configured to correct the pixel        data on a basis of the reference data.

Note that the effects described in the present specification are merelyexamples, and not limitative; other effects may be exhibited.

REFERENCE SIGNS LIST

-   100 imaging device-   101 optical unit-   102 image sensor-   103 DSP circuit-   201 pixel array portion-   202 control unit-   203 row scanning circuit-   204 reference signal generation unit-   205 input control unit-   206 analog signal processing unit-   207 AD conversion unit-   208 column scanning circuit-   209 digital signal processing unit-   301 pixel-   311 logarithmic conversion circuit-   312 voltage conversion circuit-   321 ADC-   401 input control unit-   402 correction processing unit-   403 event detection unit-   404 mode control unit-   405 exposure control unit-   406 image processing unit-   411 calibration unit-   412 correction unit-   421 light amount characteristic estimation unit-   422 reference signal characteristic estimation unit-   423 correction function generation unit-   501 temperature detection element-   511 thermal array portion-   PB pixel block

The invention claimed is:
 1. An image sensor comprising: a pixel arrayin which a plurality of pixels are disposed and which generates a pixelsignal; a reference signal generator configured to generate a referencesignal for calibration; an analog digital (AD) conversion circuitconfigured to execute AD conversion on the pixel signal and thereference signal to generate pixel data and reference data; andcorrection processing circuitry configured to correct the pixel data ona basis of the reference data, wherein the correction processingcircuitry is configured to generate a correction function on the basisof the reference data and on a basis of a reference signalcharacteristic indicating a correspondent relation between an amount ofincident light of the pixels and the reference signal and a measurementresult of the reference data with regard to the reference signal with apredetermined level, and correct the pixel data on a basis of thecorrection function.
 2. The image sensor according to claim 1, whereinthe correction processing circuitry is further configured to estimatethe reference signal characteristic on a basis of a light amountcharacteristic indicating a relation between the amount of incidentlight of the pixels and the pixel data and the measurement result of thereference data with regard to the reference signal with thepredetermined level.
 3. The image sensor according to claim 2, whereinthe correction processing circuitry is further configured to estimatethe light amount characteristic on a basis of a measurement result ofthe pixel data with respect to a predetermined amount of incident light.4. The image sensor according to claim 1, wherein the correctionprocessing circuitry is configured to update the correction function onthe basis of the reference data whenever a predetermined condition issatisfied.
 5. The image sensor according to claim 1, wherein thecorrection function is a function of correcting a variation, between thepixels, in a light amount characteristic indicating a relation betweenan amount of incident light of the pixels and the pixel data.
 6. Theimage sensor according to claim 1, further comprising: an analog signalprocessing circuit configured to execute analog signal processing on thepixel signal and the reference signal, wherein the AD conversion circuitis configured to execute the AD conversion on the pixel signal and thereference signal subjected to the analog signal processing.
 7. An imagesensor comprising: a pixel array in which a plurality of pixels aredisposed and which generates a pixel signal; a reference signalgenerator configured to generate a reference signal for calibration; ananalog digital (AD) conversion circuit configured to execute ADconversion on the pixel signal and the reference signal to generatepixel data and reference data; correction processing circuitryconfigured to correct the pixel data on a basis of the reference data;and an analog signal processing circuit configured to execute analogsignal processing on the pixel signal and the reference signal, whereinthe AD conversion circuit is configured to execute the AD conversion onthe pixel signal and the reference signal subjected to the analog signalprocessing, wherein the analog signal processing includes a logarithmicconversion process for the pixel signal and the reference signal, andthe correction processing circuitry is configured to generate acorrection function of correcting a relation between an amount ofincident light of the pixels and the pixel data to a linear relation onthe basis of the reference data, and correct the pixel data on a basisof the correction function.
 8. An image sensor comprising: a pixel arrayin which a plurality of pixels are disposed and which generates a pixelsignal; a reference signal generator configured to generate a referencesignal for calibration; an analog digital (AD) conversion circuitconfigured to execute AD conversion on the pixel signal and thereference signal to generate pixel data and reference data; correctionprocessing circuitry configured to correct the pixel data on a basis ofthe reference data; and mode control circuitry configured to set a firstdriving mode and a second driving mode in which the pixel signal outputfrom the pixel array is reduced more than in the first driving mode,wherein the correction processing circuitry is configured to correct thepixel data on the basis of the reference data during the second drivingmode.
 9. The image sensor according to claim 8, wherein the correctionprocessing circuitry is configured to generate a correction function onthe basis of the reference data during the second driving mode, andcorrect the pixel data on a basis of the correction function during thesecond driving mode.
 10. The image sensor according to claim 9, whereinthe correction processing circuitry is configured to update thecorrection function on the basis of the reference data whenever apredetermined condition is satisfied during the second driving mode. 11.The image sensor according to claim 8, further comprising: exposurecontrol circuitry configured to control exposure in the first drivingmode on a basis of the corrected pixel data.
 12. The image sensoraccording to claim 11, wherein the exposure control circuitry isconfigured to control the exposure before the second driving modetransitions to the first driving mode.
 13. The image sensor according toclaim 8, wherein the mode control circuitry is configured to set thefirst driving mode in a case in which a predetermined event is detectedon a basis of the corrected pixel data in the second driving mode. 14.The image sensor according to claim 8, wherein the pixel signal obtainedby adding signals from a plurality of the pixels is generated in thesecond driving mode.
 15. The image sensor according to claim 8, whereinthe pixels outputting the pixel signal are reduced in the second drivingmode.
 16. A signal processing device comprising: correction processingcircuitry configured to correct pixel data obtained when an analogdigital (AD) conversion circuit executes AD conversion on a pixel signalgenerated by a pixel array in which a plurality of pixels are disposed,on a basis of reference data obtained when the AD conversion unitexecutes the AD conversion on a reference signal for calibration,wherein the correction processing circuitry is configured to generate acorrection function on the basis of the reference data and on a basis ofa reference signal characteristic indicating a correspondent relationbetween an amount of incident light of the pixels and the referencesignal and a measurement result of the reference data with regard to thereference signal with a predetermined level, and correct the pixel dataon a basis of the correction function.
 17. A signal processing methodcomprising: correcting pixel data obtained when an analog digital (AD)conversion circuit executes AD conversion on a pixel signal generated bya pixel array in which a plurality of pixels are disposed, on a basis ofreference data obtained when the AD conversion unit executes the ADconversion on a reference signal for calibration, wherein correctingpixel data includes generating a correction function on the basis of thereference data and on a basis of a reference signal characteristicindicating a correspondent relation between an amount of incident lightof the pixels and the reference signal and a measurement result of thereference data with regard to the reference signal with a predeterminedlevel; and correcting the pixel data on a basis of the correctionfunction.
 18. An electronic device comprising: an image sensor; andsignal processing circuitry configured to process a signal output fromthe image sensor, wherein the image sensor includes a pixel array inwhich a plurality of pixels are disposed and which generates a pixelsignal, a reference signal generator configured to generate a referencesignal for calibration, an analog digital (AD) conversion circuitconfigured to execute AD conversion on the pixel signal and thereference signal to generate pixel data and reference data, andcorrection processing circuitry configured to correct the pixel data ona basis of the reference data, wherein the correction processingcircuitry is configured to generate a correction function on the basisof the reference data and on a basis of a reference signalcharacteristic indicating a correspondent relation between an amount ofincident light of the pixels and the reference signal and a measurementresult of the reference data with regard to the reference signal with apredetermined level, and correct the pixel data on a basis of thecorrection function.